python 3.10.13
requests 2.31.0
I'm trying to automate the upload of a file. The interface is designed for human interaction, but for every other functionality after login, we can automate interaction with the platform by noting urls and payload requirements in the developer tools interface, though it is at times complex. Uploading a file, however, has so far stumped us.
In the request header, I have (taken from the dev tools network records)
'Content-Type': 'multipart/form-data; boundary=----WebKitFormBoundaryAj4wnKXvQ3MwhGWT'
And among other fields in the payload, there's a 'batchID' key, which gets set to a value of '0'.
I've tried both
with open(xlsx_file, 'rb') as xfile:
response = requests.request(
"POST"
, url
, headers=header
, json=payload
, files={'chooser': xfile.read()}
)
and
xfile = {"chooser": open(xlsx_file, 'rb')}
response = requests.request(
"POST"
, url
, headers=header
, json=payload
, files=xfile
In both cases, I get a 200 status_code back, but the response text indicates that it's apparently not parsing the json (the java.lang.IllegalArgumentException message):
<html><body>
<script type="text/javascript">
try{
if(window.parent.dwr){
var dwr=window.parent.dwr._[0];
dwr.engine.transport.iframe.remote.beginIFrameResponse(this.frameElement);
//#DWR-REPLY
dwr.engine.remote.handleBatchException({ name:'java.lang.IllegalArgumentException', message:'Failed to find parameter: batchId (check server log for more info).' }, null);
dwr.engine.transport.iframe.remote.endIFrameResponse();
dwr.engine.transport.iframe.remote.endChunk(window);
}
}catch(e){}
</script>
</body></html>
Replacing the json arg with data doesn't change the result. The file key 'chooser' comes from the webpage element for the form (<input type="file" id="chooser" ...>).
I've confirmed the payload json is what's intended, and I've asked but there's nothing meaningful in the server logs as suggested in the message.
Does python requests handle the packaging, or do I need to somehow manually construct the payload using the boundary text from the 'Content-Type' header?
Are there any other tools that I could use to get a better idea how and what a browser sends? IT on the server side suggested I could perhaps try JMeter, but as near as I can tell that would take a lot of time and it's not obvious after following that rabbit hole that it will give me what we need.
Is the DWR (Direct Web Remoting) a factor?