Include authorization in a oauth2session for requests-oauthlib

Question

From reading various documents it seems like authorization is optionally required by oauth2 providers for refresh token requests. I'm working with the FitBit API that appears to require authorization.

I'm following the instructions here for refreshing a token with requests-oauthlib: https://requests-oauthlib.readthedocs.io/en/latest/oauth2_workflow.html#refreshing-tokens

Some setup code (not what I am using, but you get the idea:

>>> token = {
...     'access_token': 'eswfld123kjhn1v5423',
...     'refresh_token': 'asdfkljh23490sdf',
...     'token_type': 'Bearer',
...     'expires_in': '-30',     # initially 3600, need to be updated by you
...  }
>>> client_id = r'foo'
>>> refresh_url = 'https://provider.com/token'
>>> protected_url = 'https://provider.com/secret'

>>> # most providers will ask you for extra credentials to be passed along
>>> # when refreshing tokens, usually for authentication purposes.
>>> extra = {
...     'client_id': client_id,
...     'client_secret': r'potato',
... }

>>> # After updating the token you will most likely want to save it.
>>> def token_saver(token):
...     # save token in database / session

from requests_oauthlib import OAuth2Session
client = OAuth2Session(client_id, token=token, auto_refresh_url=refresh_url,
         auto_refresh_kwargs=extra, token_updater=token_saver)
r = client.get(protected_url)

However, with this call I'm getting:

MissingTokenError: (missing_token) Missing access token parameter.

I know my token is expired, but why isn't the refresh working?

Answer 1

The library is broken in this regard. See #379.

You can work around it something like this:

def _wrap_refresh(func):
    def wrapper(*args, **kwargs):
        kwargs['auth'] = (client_id, client_secret)
        kwargs.pop('allow_redirects', None)
        return func(*args, **kwargs)
    return wrapper

client = OAuth2Session(client_id, token=token,
                       auto_refresh_url=refresh_url, 
                       token_updater=token_saver)
client.refresh_token = _wrap_refresh(client.refresh_token)

Answer 2

Edit: There is still some useful info below but overriding the auth function means that my actual API requests are now failing (i.e. below is not a correct answer) I'm not sure how I got the one request I tried last time to work. It may have just returned an error (in json) rather than throwing an error, and I just assumed no raised error meant it was actually working. See a correct workaround by OrangeDog (until the library is fixed).

Well, I examined the FitBit server response, just before the MissingTokenError was being thrown. It turns out I was getting an error saying that the authentication was incorrect.

This is perhaps a useful point on its own to dwell on for a sec. The MissingTokenError seems to occur when the response doesn't contain the expected token. If you can debug and look at the response more closely, you may find the server is providing a bit more detail as to why your request was malformed. I went to the location of the error and added a print statement. This allowed me to see the JSON message from FitBit. Anyway, this approach may be useful for others getting the MissingTokenError.

    if not 'access_token' in params:
        print(params)
        raise MissingTokenError(description="Missing access token parameter.")

Anyway, after some further debugging, the authentication was not set. Additionally my client id and secret were being posted in the body (this may not have been a problem). In the FitBit examples, the client id and secret were not posted in the body but were passed via authentication. So I needed the client to pass authentication to FitBit.

So then the question was, how do I authenticate. The documentation is currently lacking in this respect. However, looking at the session object I found a .auth property that was being set and a reference to an issue (#278). In that issue a workaround is provided (shown below with my code) for manual authentication setting: https://github.com/requests/requests-oauthlib/issues/278

Note, the oauth session inherits from the requests session, so for someone that knows requests really well, this may be obvious ...

Anyway, the solution was just to set the auth parameter after initializing the session. Since FitBit doesn't need the client id and secret in the body, I removed passing in the extras as well (again, this may be a minor issue and not really impact things):

import os
import json
from requests.auth import HTTPBasicAuth
from requests_oauthlib import OAuth2Session

client_id = ""
client_secret = ""

with open("tokens.json", "r") as read_file:
    token = json.load(read_file)       

save_file_path = os.path.abspath('tokens.json')

refresh_url = 'https://api.fitbit.com/oauth2/token'

def token_saver(token):    
    with open(save_file_path, "w") as out_file:
        json.dump(token, out_file, indent = 6) 

#Note, I've removed the 'extras' input
client = OAuth2Session(client_id, token=token, auto_refresh_url=refresh_url, token_updater=token_saver)
#This was the magic line ...
auth = HTTPBasicAuth(client_id, client_secret)
client.auth = auth

url = 'https://api.fitbit.com/1.2/user/-/sleep/date/2021-01-01-2021-01-23.json'
wtf = client.get(url)

OK, I think I copied that code correctly, it is currently a bit of a mess on my end. The key part was simply the line of:

client.auth = auth

After the client was initiated.

Note, my token contains an expires_at field. I don't think the session handles the expires_in in terms of exact timing. In other words, I think expires_in only causes a refresh if its value is less than 0. I don't think it looks at the time the object was created and starts a timer or sets a property to know what expires_in is relative to. The expires_at field on the other hand seems to provide (I think) a field that is checked to ensure that the token hasn't expired at the time of the request, since expires_at is a real world, non-relative, time. Here's my token dict (with fake tokens and user_id):

{'access_token': '1234',
 'expires_in': 28800,
 'refresh_token': '5678',
 'scope': ['heartrate',
  'profile',
  'settings',
  'nutrition',
  'location',
  'weight',
  'activity',
  'sleep',
  'social'],
 'token_type': 'Bearer',
 'user_id': 'abcd',
 'expires_at': 1611455442.4566112}