I am checking a csv file of urls for the status code returning the custom object. After that I want to write it to a file. I have this working correctly, but I want to format the table else the urls are cut off. I got it to work but it doesn't append it to a new line. Also once I get this working I would like to sort by code status so I can get all the 404s to the top.

I have tried format-table and a bunch of other things

[pscustomobject]@{
Code = $statusCode
Date = Get-Date -f yyyyMMdd
URL  = $url
} | Format-Table - Autosize

This works, but doesnt create a new line each time as it does without.

Add-Type -AssemblyName System.Web
Add-Type -AssemblyName System.Web.Extensions
$inputPath = "\\Scripts\URLS\test.csv"
$outputPath = "\\Scripts\Results\statusCodeResults.txt"
$urlArray = Import-Csv -Path $inputPath | Select -ExpandProperty urls

function Get-WebStatus {  
  param($urlArray)

  foreach ($url in $urlArray) {
    try {
      $request = Invoke-WebRequest -Uri $url -Method GET
      $statusCode = $request.StatusCode
    }
    catch {
     $statusCode  = $_.Exception.Response.StatusCode.value__
    }

[pscustomobject]@{
Code = $statusCode
Date = Get-Date -f yyyyMMdd
URL  = $url
} 
}
}
Get-WebStatus -urlArray $urlArray | Out-File $outputPath

Would like it to look like this but with the URLs showing complete and then eventually sort it as well putting all 404s on top.

 Code Date     URL                                             
 ---- ----     ---                                             
 200 20190404 bsca.com/provider/account-tools/login/home.jhtml
 200 20190404 bsca.com/provider/home.jhtml                    
 200 20190404 bsca.com/provider/eligibility-benefits/eligib...
 200 20190404 bsca.com/provider/eligibility-benefits/home.j...
 200 20190404 bsca.com/provider/claims/home-auth.jhtml        
 200 20190404 bsca.com/provider/guidelines-resources/patien...
 200 20190404 bsca.com/provider/claims/search/home.jhtml      
 200 20190404 bsca.com/provider/claims/policies-guidelines/...
 404 20190404 bsca.com/provider/claims/view-rationale/home....
 200 20190404 bsca.com/provider/guidelines-resources/home.j...

1 Answers

1
Vasiliy Kovalchuk On Best Solutions

As mentioned in the comments, direct blind answer to your question is to use -Width parameter of Out-File cmdlet with any suitable value for your needs, e.g.:

Get-WebStatus -urlArray $urlArray | Out-File $outputPath -Width 200

And for that weird king of semi-bubble sorting you could combine two filtered arrays like this:

$url_responses_sorted = @(
    $url_responses | Where-Object -Property Code -EQ 404
    $url_responses | Where-Object -Property Code -NE 404
)

But I encourage you to consider saving your [PSCustomObject] array in CSV format; that way you could any time load your data and work with it using a wide variety of languages/instruments.

Export-Csv -Path $outputPath

Speaking of PowerShell, you can load your data by Import-Csv cmdlet similarly to how you load your $urlArray, and then sort it, group it and filter what you need like this:

PS C:\> $url_responses = Get-WebStatus -urlArray $urlArray
PS C:\> $url_responses

Code Date     URL
---- ----     ---
200  20190405 https://stackoverflow.com/questions/55521742/issue-with-powershel-pscustomobject-only-writing-one-line...
404  20190405 https://google.com/non-existing-page
200  20190405 https://github.com/
0    20190405 https://non-existing.url/
404  20190405 https://google.com/non-existing-page-2

PS C:\> $url_responses | Sort-Object -Property Code -Descending

Code Date     URL
---- ----     ---
404  20190405 https://google.com/non-existing-page-2
404  20190405 https://google.com/non-existing-page
301  20190405 https://google.com/
200  20190405 https://stackoverflow.com/questions/55521742/issue-with-powershel-pscustomobject-only-writing-one-line...
0    20190405 https://non-existing.url/

PS C:\> $url_responses | Group-Object -Property Code

Count Name                      Group
----- ----                      -----
    2 200                       {@{Code=200; Date=20190405; URL=https://stackoverflow.com/questions/55521742/issue-w...
    1 301                       {@{Code=301; Date=20190405; URL=https://google.com/}}
    2 404                       {@{Code=404; Date=20190405; URL=https://google.com/non-existing-page}, @{Code=404; D...
    1 0                         {@{Code=0; Date=20190405; URL=https://non-existing.url/}}

PS C:\> $url_responses | Where-Object -Property Code -EQ 404

Code Date     URL
---- ----     ---
404  20190405 https://google.com/non-existing-page
404  20190405 https://google.com/non-existing-page-2

Other considerations (on using Invoke-WebRequest)

  1. if the webservers from your URL list support HEAD requests, than you'll benefit in speed from using that method istead of GET:
    $response = Invoke-WebRequest -Uri $Uri -Method HEAD
    
  2. if you also interested in redirection codes, than I suggest to use -MaximumRedirection 0 with -ErrorAction SilentlyContinue (that way script suppress error message about exceeding of the maximum redirection count, which exactly our intention)
    $response = Invoke-WebRequest -Uri $Uri -Method HEAD -MaximumRedirection 0 -ErrorAction SilentlyContinue
    
  3. bandwidths are fat nowdays, so we don't need poping out download progress bar for each request. Where is no any parameter that you could pass to the Invoke-WebRequest to suppress progress, but we could temporarily disable it globally like this:
    $progressPreference = 'silentlyContinue'
    $response = Invoke-WebRequest -Uri $Uri -Method HEAD -MaximumRedirection 0 -ErrorAction SilentlyContinue
    $progressPreference = 'Continue'
    
    of course in your case you should place assigning to that preference variable outside of your loop.