Using access token generated from googleapis.com/oauth2/v4/token, returns UNAUTHENTICATED

Question

I am trying to fetch my google photos albums in my website. Now to do that I am using the Google Photos API.

The problem is, I can successfully fetch an access token, but using the access token to call the google API fails with an UNAUTHENTICATED error.

Now, this is where it gets weirder... When I run the same steps in postman, with fetching the access token and using that access token to call the google photos API, it works properly. Also, if I use the access token generated by postman in my code, it works, but using the access token from my program in postman, and postman fails.

So I am not sure what went wrong where. I would appreciate any help here. Please someone help.

The following is my code

I use https://www.googleapis.com/oauth2/v4/token to create an access token

fetch("https://www.googleapis.com/oauth2/v4/token", {
    method: "POST",
    headers: {
      "Content-Type": "text/plain",
    },
    body: JSON.stringify({
      grant_type: "refresh_token",
      client_id: "<client_id>",
      client_secret: "<client_secret>",
      refresh_token: "<refresh_token>",
    }),
  })

Now of course to get the client id and secret, we create a google dev account, create an app, and create oauth2 credentials in there and then i got it. Also in the App, you search for the required APIs that you want to use and you allow it. In my case I allowed, photos and drive API To get the refresh token, I went to the oauth playground and generated it from there. Now oauth playground has covered the user consent part of it. In the oauth playground also, i selected the correct scopes for drive and photos API

Now I know all my steps are correct until here, because, the response i get has the correct schema, with the access_token, the expiry and the correct scopes I mentioned above.

The problem is the part after here. When I try to use this token as below

const Authorization = `Bearer ${token}`;
const res = await fetch("https://photoslibrary.googleapis.com/v1/albums", {
  headers: {
    "Content-Type": "application/json",
    Authorization,
  },
});

It returns an error

{
    "code": 401,
    "message": "Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
    "status": "UNAUTHENTICATED"
}