curl vs python "requests" when hitting APIs

Question

I am trying to hit the Bitbucket API for my account, and a successful attempt looks like:

curl --user screename:mypassword https://api.bitbucket.org/1.0/user/repositories

in the command line. In python, I try:

import requests
url = 'https://api.bitbucket.org/1.0/user/repositories'

then

r = requests.post(url, data={'username': myscreename, 'password':mypassword})

and

r = requests.post(url, data="myscreename:mypassword")

and

r = requests.post(url, data={"user": "myscreename:mypassword"})

all get 405 error. The API is https://confluence.atlassian.com/bitbucket/rest-apis-222724129.html.

I wonder:

1. What am I doing wrong in the requests version, they all look similar to my curl attempt

2. What is the difference between requesting with curl and python requests module? What general pattern can I recognize when reading an API with a curl example and then writing it in python?

Thank you

ANSWER:

it required the right headers

https://answers.atlassian.com/questions/18451025/answers/18451117?flashId=-982194107

UPDATE:

# ===============
# get user
# ===============
import requests
import json

# [BITBUCKET-BASE-URL], i.e.: https://bitbucket.org/
url = '[BITBUCKET-BASE-URL]/api/1.0/user/'
headers = {'Content-Type': 'application/json'}

# get user
# [USERNAME], i.e.: myuser
# [PASSWORD], i.e.: itspassword
r = requests.get(url, auth=('[USERNAME]', '[PASSWORD]'), headers=headers)
print(r.status_code)
print(r.text)
#print(r.content)

Answer 1

Here's a way to do basic HTTP auth with Python's requests module:

requests.post('https://api.bitbucket.org/1.0/user/repositories', auth=('user', 'pass'))

With the other way you're passing the user/pass through the request's payload, which is not desired since HTTP basic auth has its own place in the HTTP protocol.

If you want to "see" what's happening under the hood with your request I recommend using httpbin:

>>> url = "http://httpbin.org/post"
>>> r = requests.post(url, data="myscreename:mypassword")
>>> print r.text
{
  "args": {}, 
  "data": "myscreename:mypassword", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate", 
    "Content-Length": "22", 
    "Host": "httpbin.org", 
    "User-Agent": "python-requests/2.5.1 CPython/2.7.6 Darwin/14.3.0"
  }, 
  "json": null, 
  "origin": "16.7.5.3", 
  "url": "http://httpbin.org/post"
}

>>> r = requests.post(url, auth=("myscreename", "mypassword"))
>>> print r.text
{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate", 
    "Authorization": "Basic bXlzY3JlZW5hbWU6bXlwYXNzd29yZA==", 
    "Content-Length": "0", 
    "Host": "httpbin.org", 
    "User-Agent": "python-requests/2.5.1 CPython/2.7.6 Darwin/14.3.0"
  }, 
  "json": null, 
  "origin": "16.7.5.3", 
  "url": "http://httpbin.org/post"
}

And with curl:

curl -X POST --user myscreename:mypassword http://httpbin.org/post
{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Authorization": "Basic bXlzY3JlZW5hbWU6bXlwYXNzd29yZA==", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/7.37.1"
  }, 
  "json": null, 
  "origin": "16.7.5.3", 
  "url": "http://httpbin.org/post"
}

Notice the resemblance between the last python example and the cURL one.

Now, getting right the API's format is another story, check out this link: https://answers.atlassian.com/questions/94245/can-i-create-a-bitbucket-repository-using-rest-api

The python way should be something like this:

requests.post('https://api.bitbucket.org/1.0/repositories', auth=('user', 'pass'), data = "name=repo_name")

Answer 2

With python3, you can use json={...} instead of data={...}, and it will set the header automatically to application/json:

import requests
url = 'https://api.bitbucket.org/1.0/user/repositories'

data = {
    'data1': 'asd',
    'data2': 'asd'
}
req = requests.post(url, auth=('user', 'password'), json = data)
data = req.json()
# data['index']