How to get all the extensions of an url address given the base url address?

93 views Asked by At

Is there a way to know all the extensions of a url?

For example let's say there is a website called www.blabla.com

and there is another page www.blabla.com/blabla2.html.

Is there a way to know www.blabla.com/blabla2.html from www.blabla.com only?

2

There are 2 answers

0
thesquaregroot On

I think what you're looking for is a web crawler: https://en.wikipedia.org/wiki/Web_crawler

At a high level, you request the base page and then request all of the pages linked on that page. Continuing this process should yield all of the URLs that are referenced on the website. However, if www.blabla.com/blabla2.html is real page but is never actually linked to on the website, this process would not find it.

2
Diversity On

No it is not possible. Because the names behind the main domain are mostly complete virtual. Means the resource behind that name must not be a concrete file or a concrete web page

You can't know it until you visited each url at least for one time. Than the WebCrawler mention by another answer makes sense.

My answer also implies that there is now malware to checkout whats goin on.

Look at the Unix wget. This offers some nice possibilities.