Invoke-WebRequest working after server is unreachable

Question

I am using Invoke-WebRequest in a SCOM PowerShell script to periodically monitor the availability of a URI. My script is fairly simple (since I have very little knowledge of PS :-) ):

$scomapi = new-object -comObject "MOM.ScriptAPI"
$scompb = $scomapi.CreatePropertyBag()
$fullHostName = "https://" + <full path to monitored web endpoint>
$result = Invoke-WebRequest $fullHostName
if($result.content) {
    $scompb.AddValue("ConfigurationReachable",$true);
} else {
    $scompb.AddValue("ConfigurationReachable",$false);
}           
$scomapi.AddItem($scompb) 
$scomapi.ReturnItems()

In order to test this script, I did manual changes in the hosts file on the client running the SCOM agent where I want to do the monitoring. Interestingly, the script succeeds in fetching the web endpoint even after the host is unreachable (tested this by pinging from that machine).

I did some further tests directly from the command line, and nothing changes. Even though I have no ping to the remote address, Invoke-WebRequest still succeeds and fetches the web page. So what am I doing wrong here?

Answer 1

Without testing it, I'd guess it's dns caching.

The powershell-session probably caches the ip on the first request and ignores your hosts-file update(just uses the old working ip).

Try running your script before and after disconnecting your network adapter/cable to simulate server failure.

UPDATE: What I'm trying to say above is that the script would work perfectly if the servers is unavailable, but your simulation using the hosts-file is giving a "false-positive" (so ignore the results).

If you really need to test the script with editing the hosts file, disable .Net dns cache in the session by adding the following line at the start of your script:

[System.Net.ServicePointManager]::DnsRefreshTimeout = 0

Answer 2

Per discussion in the comments the issue is caching; only it's not the IP being cached that's the problem (at least, not the only problem); the content is also being cached; so instead of going to the web server to get your resource, the system's cheating and getting it locally. You can prevent this by adding -Headers @{"Cache-Control"="no-cache"} to your invoke-webrequest.

See example test script below; try running with and without the cache-control header before and after your hosts file tweak.

cls

$urlHost = 'server.mydomain.com'
$endpointUrl = ("https://{0}/path/to/resource.jpg" -f $urlHost)

#can be set once at the start of the script
[System.Net.ServicePointManager]::DnsRefreshTimeout = 0

#I don't have Clear-DnsClientCache; but the below should do the same thing
#Should be called inside any loop before the invoke-webrequest to ensure
#flush your machine's local dns cache each time
ipconfig /flushdns

#prove that our hosts update worked:
#optional, but will help in debugging
Test-Connection $urlHost -Count 1 | select ipv4address

#ensure we don't have a remembered result if invoke-request is in a loop
$result = $null
#make the actual call
#NB: -headers parameter takes a value telling the system not to get anything
#from the cache, but rather to send the request back to the root source.
$result = Invoke-WebRequest $endpointUrl -Headers @{"Cache-Control"="no-cache"}

#output the result; 200 means all's good (google http status codes for info on other values)
("`nHTTP Status Code: {0}`n" -f $result.StatusCode)

#output actual result; optional, but may be useful to see what's being returned (e.g. is it your page/image, or a 404 page / something unexpected
$result

Answer 3

I know this is old, but just in case this helps someone:

I ran across a similar issue, and adding "-Disable KeepAlive" resolved it for me.