learning CTF games, I made gobuster from git.
Using it by
./gobuster dir -u http://10.10.19.14:3333 -w ~/HTools/lists/gobuster_big.txt
brings following output:
===============================================================
Gobuster v3.1.0
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@firefart)
===============================================================
[+] Url: http://10.10.19.14:3333
[+] Method: GET
[+] Threads: 10
[+] Wordlist: /home/jens/HTools/lists/gobuster_big.txt
[+] Negative Status codes: 404
[+] User Agent: gobuster/3.1.0
[+] Timeout: 10s
===============================================================
2020/11/20 13:52:49 Starting gobuster in directory enumeration mode
===============================================================
And then starts a lot of rubbish. Tons of blanks or eol. And within paragraphs of
/<meta name="twitter:image:src" content="https://avatars1.githubusercontent.com/u/236661?s=400&v=4" /><meta name="twitter:site" content="@github" /><meta name="twitter:card" content="summary" /><meta name="twitter:title" content="digination/dirbuster-ng" /><meta name="twitter:description" content="dirbuster-ng is C CLI implementation of the Java dirbuster tool - digination/dirbuster-ng" /> (Status: 400) [Size: 301]
/<meta property="og:image" content="https://avatars1.githubusercontent.com/u/236661?s=400&v=4" /><meta property="og:site_name" content="GitHub" /><meta property="og:type" content="object" /><meta property="og:title" content="digination/dirbuster-ng" /><meta property="og:url" content="https://github.com/digination/dirbuster-ng" /><meta property="og:description" content="dirbuster-ng is C CLI implementation of the Java dirbuster tool - digination/dirbuster-ng" /> (Status: 400) [Size: 301]
/<meta name="optimizely-datafile" content="{"version": "4", "rollouts": [], "typedAudiences": [], "anonymizeIP": true, "projectId": "16737760170", "variables": [], "featureFlags": [], "experiments": [{"status": "Running", "audienceIds": [], "variations": [{"variables": [], "id": "18630402174", "key": "launchpad"}, {"variables": [], "id": "18866331456", "key": "control"}], "id": "18651193356", "key": "_features_redesign_rollout", "layerId": "18645992876", "trafficAllocation": [{"entityId": "18630402174", "endOfRange": 500}, {"entityId": "18866331456", "endOfRange": 1000}, {"entityId": "18630402174", "endOfRange": 5000}, {"entityId": "18630402174", "endOfRange": 5500}, {"entityId": "18866331456", "endOfRange": 10000}], "forcedVariations": {"143327983.1601483920": "launchpad", "1955030087.1562868941": "launchpad", "1983887325.1550021416": "launchpad", "1947530619.1600461583": "launchpad"}}, {"status": "Running", "audienceIds": [], "variations": [{"variables": [], "id": "19157301901", "key": "launchpad"}, {"variables": [], "id": "19115494094", "key": "control"}], "id": "19139621248", "key": "signup_prompt_launchpad", "layerId": "19159450734", "trafficAllocation": [{"entityId": "19115494094", "endOfRange": 5000}, {"entityId": "19157301901", "endOfRange": 10000}], "forcedVariations": {"262350301.1605284875": "launchpad", "550830608.1557172966": "launchpad"}}], "audiences": [{"conditions": "[\"or\", {\"match\": \"exact\", \"name\": \"$opt_dummy_attribute\", \"type\": \"custom_attribute\", \"value\": \"$opt_dummy_value\"}]", "id": "$opt_dummy_audience", "name": "Optimizely-Generated Audience for Backwards Compatibility"}], "groups": [], "attributes": [{"id": "16822470375", "key": "user_id"}, {"id": "17143601254", "key": "spammy"}, {"id": "18175660309", "key": "organization_plan"}, {"id": "18813001570", "key": "is_logged_in"}, {"id": "19073851829", "key": "geo"}], "botFiltering": false, "accountId": "16737760170", "events": [{"experimentIds": [], "id": "17911811441", "key": "hydro_click.dashboard.teacher_toolbox_cta"}, {"experimentIds": [], "id": "18124116703", "key": "submit.organizations.complete_sign_up"}, {"experimentIds": [], "id": "18145892387", "key": "no_metric.tracked_outside_of_optimizely"}, {"experimentIds": [], "id": "18178755568", "key": "click.org_onboarding_checklist.add_repo"}, {"experimentIds": [], "id": "18180553241", "key": "submit.repository_imports.create"}, {"experimentIds": [], "id": "18186103728", "key": "click.help.learn_more_about_repository_creation"}, {"experimentIds": [], "id": "18188530140", "key": "test_event.do_not_use_in_production"}, {"experimentIds": [], "id": "18191963644", "key": "click.empty_org_repo_cta.transfer_repository"}, {"experimentIds": [], "id": "18195612788", "key": "click.empty_org_repo_cta.import_repository"}, {"experimentIds": [], "id": "18210945499", "key": "click.org_onboarding_checklist.invite_members"}, {"experimentIds": [], "id": "18211063248", "key": "click.empty_org_repo_cta.create_repository"}, {"experimentIds": [], "id": "18215721889", "key": "click.org_onboarding_checklist.update_profile"}, {"experimentIds": [], "id": "18224360785", "key": "click.org_onboarding_checklist.dismiss"}, {"experimentIds": [], "id": "18234832286", "key": "submit.organization_activation.complete"}, {"experimentIds": [], "id": "18252392383", "key": "submit.org_repository.create"}, {"experimentIds": [], "id": "18257551537", "key": "submit.org_member_invitation.create"}, {"experimentIds": [], "id": "18259522260", "key": "submit.organization_profile.update"}, {"experimentIds": [], "id": "18564603625", "key": "view.classroom_select_organization"}, {"experimentIds": [], "id": "18568612016", "key": "click.classroom_sign_in_click"}, {"experimentIds": [], "id": "18572592540", "key": "view.classroom_name"}, {"experimentIds": [], "id": "18574203855", "key": "click.classroom_create_organization"}, {"experimentIds": [], "id": "18582053415", "key": "click.classroom_select_organization"}, {"experimentIds": [], "id": "18589463420", "key": "click.classroom_create_classroom"}, {"experimentIds": [], "id": "18591323364", "key": "click.classroom_create_first_classroom"}, {"experimentIds": [], "id": "18591652321", "key": "click.classroom_grant_access"}, {"experimentIds": [], "id": "18607131425", "key": "view.classroom_creation"}, {"experimentIds": [], "id": "18831680583", "key": "upgrade_account_plan"}, {"experimentIds": ["19139621248"], "id": "19064064515", "key": "click.signup"}, {"experimentIds": [], "id": "19075373687", "key": "click.view_account_billing_page"}, {"experimentIds": ["19139621248"], "id": "19077355841", "key": "click.dismiss_signup_prompt"}, {"experimentIds": [], "id": "19079713938", "key": "click.contact_sales"}, {"experimentIds": [], "id": "19120963070", "key": "click.compare_account_plans"}, {"experimentIds": [], "id": "19151690317", "key": "click.upgrade_account_cta"}], "revision": "319"}" /> (Status: 414) [Size: 323]
[ERROR] 2020/11/20 13:52:49 [!] parse http://10.10.19.14:3333/<span style="background-color: #79b8ff;width: 0%;" class="Progress-item progress-pjax-loader-bar "></span>: invalid URL escape "%;\""
Progress: 149 / 17906 (0.83%)
or
/<svg height="32" class="octicon octicon-mark-github text-white" viewBox="0 0 16 16" version="1.1" width="32" aria-hidden="true"><path fill-rule="evenodd" d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"></path></svg> (Status: 403) [Size: 1132]
I tried quiet mode. No help. Using Ubuntu 20.04.
Why is that? Thx
Your wordlist file might have the wrong format.
It should contain only a list of dir names written in column because gobuster does the parsing of the specified file line-by-line, inserts the entire line content to the URI and makes a request. For example your worldlist file should look like this:
It seems that when you did the download of this file you've saved it with the entire HTML markup.