Potentially dangerous Request.Path due to doubled url

Question

I'm maintaining a legacy .NET 4.7.2 MVC application that uses Umbraco CMS 7.15.7. We started getting failed requests registered in Azure Application Insights with the following message

A potentially dangerous Request.Path value was detected from the client (:).

I've seen that it is related to requests such as these (below is a sample request that usually occurs in Application Insights logs, I've just changed the real website domain):

timestamp [UTC]
2021-09-28T21:50:31.1808953Z

name
GET /companies/ https:/www.sample-site.com/companies/sample-company

url
https://www.sample-site.com/companies/ https:/www.sample-site.com/companies/sample-company

success
False

resultCode
400

duration
0.1136

performanceBucket
<250ms

itemType
request
customDimensions
{"_MS.ProcessedByMetricExtractors":"(Name:'Requests', Ver:'1.1')"}

operation_Name
GET /companies/ https:/www.sample-site.com/companies/sample-company

client_Type
PC

client_IP
0.0.0.0

sdkVersion
web:2.9.0-23612

I realise that the problem (:) is because the url and the name end up being somehow "doubled" (https://www.sample-site.com/companies/ https:/www.sample-site.com/companies/sample-company). My goal is not to allow such requests, but rather find the cause of them.

Here are my conclusions as to what might be causing this issue:

• The failed requests seem to occur periodically, and in bulk - every 5-10 hours. One product page never gets called twice within a bulk. This led me to believe that they might be caused by a bot, but then again the pattern isn't always clear - there will be minutes apart between two requests.
• The requests seem to occur mostly for product pages that are under a certain node in Umbraco (in this instance, the companies node). However, there are some product pages under that node in Umbraco that don't have the occurrence.
• The failed request contains https:/ with only one "/", which is odd and seems that this part is concatenated at some point
• The few successful requests under the companies node always have a 301 status code, which leads me to believe that the issue could be related to a broken redirect rule somewhere
• The error started occurring after we changed some content in Umbraco for certain product pages (mostly canonicalUrl properties of product page nodes). But then again, there are some product pages that do have the property filled out, but do not get the issue.
• I can never relate these requests to a user or session in Application Insights

I am aware that this might be too specific to debug and that this might be caused by practically anything - but I do have a feeling that it's somehow related to Umbraco, so somebody with experience in Umbraco and the knowledge how the name property could end up being generated as GET /companies/ https:/www.sample-site.com/companies/sample-company could lead me to some clues?

As additional information, here is a successfully executed request to a product page under the same node in Umbraco:

timestamp [UTC]
2021-09-29T07:27:54.2739984Z

name
GET /companies/second-company

url
https://www.sample-site.com/companies/second-company

success
True

resultCode
301

duration
6.4651000000000005

performanceBucket
<250ms

itemType
request
customDimensions
{"_MS.ProcessedByMetricExtractors":"(Name:'Requests', Ver:'1.1')"}

operation_Name
GET /companies/second-company

client_Type
PC

client_IP
0.0.0.0

sdkVersion
web:2.9.0-23612

Answer 1

Can you add ValidateRequest=false at the top of your page.

Also, As you are using .Net 4.7.2 you need to allow the below urls in web.config file.

<system.web>
    <httpRuntime 
            requestPathInvalidCharacters="&lt;,*,&gt;,%,&amp;,\,?" />
</system.web>

I have removed the (:), the original default string is

<httpRuntime 
          requestPathInvalidCharacters="&lt;,&gt;,*,%,&amp;,:,\,?" />

Check these SO1 and SO2 with related discussions.

Answer 2

After doing some research, I found the cause of the issue, so I believe it could be beneficial to others if I share it even though quite some time passed: the requests with incorrect URL were sent by an external SEO tool which we used - so the error originated outside of the system. These requests were sent by the SEO tool as they are shown here, e.g. https://www.sample-site.com/companies/ https:/www.sample-site.com/companies/sample-company, so they weren't malformed somewhere in the request pipeline of our system.