How can I use the CSS selector to select a form when there are multiple forms on a page?

Question

I am trying to fill out a specific form on a webpage but the CSS selector I am using returns an error

I have tried these separately:

 1. browser.select_form('form[method="post"]')
 2. browser.select_form()
 3. browser.select_form('form[action="/xxx"]')

But it either selects the search form at the top (no input) or returns the not found error:

in select_form raise LinkNotFoundError() mechanicalsoup.utils.LinkNotFoundError

I expected it to select the correct form because this is the CSS of the form:

<form action="/xxx" method="post" novalidate="novalidate">

EDIT: When I look through the HTML of the webpage it doesnt have the form I am looking for. But when I inspect the elements of the webpage it is there. I assume this is the issue. Why and how is this happening and how can I use MechanicalSoup to solve the problem?

Answer 1

I cannot add a comment so pardon this answer.

But without looking at the page in question, I believe this might just be a case where the form is being inserted into the html through javascript on page load, therefore does not show up in raw html.

You might want to look into this

https://pythonprogramming.net/javascript-dynamic-scraping-parsing-beautiful-soup-tutorial/

https://medium.com/ymedialabs-innovation/web-scraping-using-beautiful-soup-and-selenium-for-dynamic-page-2f8ad15efe25