Getting Contents of a Webpage Form After Submission with Python

Question

I want to learn how to submit values to a webpage form by using python. I have created the following web page

http://www.vproce.net/deu-hdp.html

I want to submit the following values to the form above. donem="303", fakulte2="107", hafta="03/10/2022_09/10/2022" and hoca="14_382"

I have used the following codes.

import mechanize,re

url = 'http://www.vproce.net/deu-hdp.html'
values = {'donem': '303',
          'fakulte2': '107',
          'hafta': '03/10/2022_09/10/2022',
          'hoca': '14_382'}

br = mechanize.Browser()
br.set_handle_robots(False)   # ignore robots
br.set_handle_refresh(False)  # can sometimes hang without this
br.addheaders = [('User-agent', 'Firefox')]
br.open( url )
br.select_form( 'hdp' )
br.form[ 'donem' ] = '303'
br.form[ 'fakulte2' ] = '107'
br.form[ 'hafta' ] = '03/10/2022_09/10/2022'
br.form[ 'hoca' ] = '14_382'
br.submit()

Does the above code submit the form as I want? I dont know how to verify it. If so, I want to retrieve the contents of the new form after submission. How can get it?

Thank you very much for your guidence. bkarpuz

Answer 1

Your code seems to be on the right track for filling out a web form, but there are a few issues that need to be addressed to ensure it works correctly. I'll provide you with an example of how to submit values to the form on the given webpage using Python with the mechanize library. However, please note that web scraping may be subject to the website's terms of use and may require permission. Here's an example code snippet to achieve your goal:

import mechanize

url = 'http://www.vproce.net/deu-hdp.html'

br = mechanize.Browser()

form_data = {
    'donem': '303',
    'fakulte2': '107',
    'hafta': '03/10/2022_09/10/2022',
    'hoca': '14_382'
}

br.open(url)
br.select_form(name='hdp')

# Fill out the form with your data
for field_name, value in form_data.items():
    br[field_name] = value

response = br.submit()

# print(response.read().decode('utf-8'))

with open('form_response.html', 'w') as file:
    file.write(response.read().decode('utf-8'))

print("Form submitted successfully. Response saved to 'form_response.html'")