I am trying to write a python script to login to the following site in order to automatically keep on eye on some account details: https://gateway.usps.com/eAdmin/view/signin

I have the right credentials, but something isn't quite working correctly, I don't know if it is because of the hidden inputs that exist on the form

import requests
from bs4 import BeautifulSoup

user='myusername'
passwd='mypassword'

s=requests.Session()

r=s.get("https://gateway.usps.com/eAdmin/view/signin")
soup=BeautifulSoup(r.content)
sp=soup.find("input",{"name":"_sourcePage"})['value']
fp=soup.find("input",{"name":"__fp"})['value']
si=soup.find("input",{"name":"securityId"})['value']

data={
  "securityId": si,
  "username":user,
  "password":passwd,
  "_sourcePage":sp,
  "__fp":fp}

headers={"Content-Type":"application/x-www-form-urlencoded",
  "Host":"gateway.usps.com",
  "Origin":"https://gateway.usps.com",
  "Referer":"https://gateway.usps.com/eAdmin/view/signin"}

login_url="https://gateway.usps.com/eAdmin/view/signin" 

r=s.post(login_url,headers=headers,data=data,cookies=r.cookies)
print(r.content)

_sourcePage, securityId and __fp are all hidden input values from the page source. I am scraping this from the page, but obviously when I get to do the POST request, I'm opening the url again, so these values change and are no longer valid. However, I'm unsure how to rewrite the POST line to ensure that I extract the correct hidden values for submission.

I don't think that this is only relevant to this site, but for any site with hidden random values.

1 Answers

0
Marko On

You can't do that.

You are trying to authenticate using an HTTP POST request outside the application scope, the login page and his own web form.

For security reasons the web page implements differents techniques, one of all the Anti CSRF Token ( which it's probably __sourcePage ) to ensure that the login request comes exclusively from the web page.

For this reason, every time you scrape the page grabbing the content of the security hidden inputs, the web application generate them every time. Thus when you reuse them to craft the final request of course they are not anymore valid.

See also: https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)