I have a proprietary Python project that I manage using Poetry. I stored the code in the Google Artifact Registry in the past, but now I switched to a different dev environment (Arch Linux in a VM -> Windows + Ubuntu WSL) and I can't upload it the AR, while getting this error (this is on my new WSL machine):

$ poetry publish --build --repository all-mpm                                                                                                      
There are 2 files ready for publishing. Build anyway? (yes/no) [no] yes
Building mpm (0.1.54)
  - Building sdist
  - Built mpm-0.1.54.tar.gz
  - Building wheel
  - Built mpm-0.1.54-py3-none-any.whl

Publishing mpm (0.1.54) to all-mpm
 - Uploading mpm-0.1.54-py3-none-any.whl FAILED

HTTP Error 401: Unauthorized | b'The request does not have valid authentication credentials.\n'

Here's the catch: This is only happens to this repository. If I create a new repository on the same machine using the same project, I don't get an error and am able to upload my package successfully.


I believe this is a keyring issue. Listing all my keyring backends:

$ keyring --list-backends   
keyrings.gauth.GooglePythonAuth (priority: 9)
keyring.backends.chainer.ChainerBackend (priority: 10)
keyring.backends.SecretService.Keyring (priority: 5)
keyring.backends.fail.Keyring (priority: 0)

I used gsutil for its creation:

gcloud artifacts repositories create my-repo \
    --repository-format=python \
    --location=europe-west3

After that I set for my Python project which I manage using poetry.

poetry config repository.myrepo "https://europe-west3-python.pkg.dev/my-project/my-repo/

My usual build process:

poetry publish --build --repository my-repo

If I run this command with the verbose option (-vvv), I get the following details:

$ poetry publish --build --repository all-mpm -vvv          
Loading configuration file /home/til/.config/pypoetry/config.toml
Loading configuration file /home/til/.config/pypoetry/auth.toml
There are 2 files ready for publishing. Build anyway? (yes/no) [no] yes
Using virtualenv: /home/til/.cache/pypoetry/virtualenvs/mpm-W0UszHAu-py3.11
Building mpm (0.1.54)
  - Building sdist
Ignoring: tests/__pycache__/test_plytix.cpython-311-pytest-7.4.2.pyc
********redacted********
  - Adding: /home/til/code/aplus-automation/src/mpm/__init__.py
********redacted********
  - Adding: pyproject.toml
  - Adding: README.md
  - Built mpm-0.1.54.tar.gz
  - Building wheel
Ignoring: htmlcov/d_a44f0ac069e85531_test_plytix_py.html
********redacted********
  - Adding: /home/til/code/aplus-automation/src/mpm/__init__.py
********redacted********
Skipping: /home/til/code/aplus-automation/LICENSE
Skipping: /home/til/code/aplus-automation/COPYING
  - Built mpm-0.1.54-py3-none-any.whl

[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
[keyring.backend] Loading Google Auth
Found authentication information for all-mpm.
Publishing mpm (0.1.54) to all-mpm
 - Uploading mpm-0.1.54-py3-none-any.whl 0%[urllib3.connectionpool] Starting new HTTPS connection (1): europe-west3-python.pkg.dev:443
 - Uploading mpm-0.1.54-py3-none-any.whl 100%[urllib3.connectionpool] https://europe-west3-python.pkg.dev:443 "POST /mpm99-398708/all-mpm HTTP/1.1" 401 60
 - Uploading mpm-0.1.54-py3-none-any.whl FAILED

  Stack trace:

  1  ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/publishing/uploader.py:265 in _upload_file
      263│                     bar.display()
      264│                 else:
    → 265│                     resp.raise_for_status()
      266│             except (requests.ConnectionError, requests.HTTPError) as e:
      267│                 if self._io.output.is_decorated():

  HTTPError

  401 Client Error: Unauthorized for url: https://europe-west3-python.pkg.dev/mpm99-398708/all-mpm

  at ~/.local/lib/python3.11/site-packages/requests/models.py:1021 in raise_for_status
      1017│                 f"{self.status_code} Server Error: {reason} for url: {self.url}"
      1018│             )
      1019│ 
      1020│         if http_error_msg:
    → 1021│             raise HTTPError(http_error_msg, response=self)
      1022│ 
      1023│     def close(self):
      1024│         """Releases the connection back to the pool. Once this method has been
      1025│         called the underlying ``raw`` object must not be accessed again.

The following error occurred when trying to handle this error:


  Stack trace:

  11  ~/.local/lib/python3.11/site-packages/cleo/application.py:327 in run
       325│ 
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a

  10  ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/console/application.py:190 in _run
       188│         self._load_plugins(io)
       189│ 
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│ 

   9  ~/.local/lib/python3.11/site-packages/cleo/application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│ 
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│ 

   8  ~/.local/lib/python3.11/site-packages/cleo/application.py:473 in _run_command
       471│ 
       472│         if error is not None:
     → 473│             raise error
       474│ 
       475│         return terminate_event.exit_code

   7  ~/.local/lib/python3.11/site-packages/cleo/application.py:457 in _run_command
       455│ 
       456│             if command_event.command_should_run():
     → 457│                 exit_code = command.run(io)
       458│             else:
       459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

   6  ~/.local/lib/python3.11/site-packages/cleo/commands/base_command.py:119 in run
       117│         io.input.validate()
       118│ 
     → 119│         status_code = self.execute(io)
       120│ 
       121│         if status_code is None:

   5  ~/.local/lib/python3.11/site-packages/cleo/commands/command.py:62 in execute
        60│ 
        61│         try:
     →  62│             return self.handle()
        63│         except KeyboardInterrupt:
        64│             return 1

   4  ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/console/commands/publish.py:82 in handle
        80│         )
        81│ 
     →  82│         publisher.publish(
        83│             self.option("repository"),
        84│             self.option("username"),

   3  ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/publishing/publisher.py:86 in publish
        84│         )
        85│ 
     →  86│         self._uploader.upload(
        87│             url,
        88│             cert=resolved_cert,

   2  ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/publishing/uploader.py:107 in upload
       105│ 
       106│         try:
     → 107│             self._upload(session, url, dry_run, skip_existing)
       108│         finally:
       109│             session.close()

   1  ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/publishing/uploader.py:191 in _upload
       189│     ) -> None:
       190│         for file in self.files:
     → 191│             self._upload_file(session, url, file, dry_run, skip_existing)
       192│ 
       193│     def _upload_file(

  UploadError

  HTTP Error 401: Unauthorized | b'The request does not have valid authentication credentials.\n'

  at ~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/poetry/publishing/uploader.py:271 in _upload_file
      267│                 if self._io.output.is_decorated():
      268│                     self._io.overwrite(
      269│                         f" - Uploading {file.name} FAILED"
      270│                     )
    → 271│                 raise UploadError(e)
      272│             finally:
      273│                 self._io.write_line("")
      274│ 
      275│     def _register(self, session: requests.Session, url: str) -> requests.Response:

Other measures I took:

  • reinstallation of gsutils
  • consulted the AR docs, setting up using a) ADC, b) a service account, c) .pypirc file
  • added the keyring to poetry

More debug info:

Poetry plugins:

$ poetry self show plugins    

  • poetry-plugin-export (1.5.0) Poetry plugin to export the dependencies to various formats
      1 application plugin

      Dependencies
        - poetry (>=1.5.0,<2.0.0)
        - poetry-core (>=1.6.0,<2.0.0)

After removing .config/gcloud/ and setting up using gcloud init again (still getting the bad credentials error)

$ gcloud init
Welcome! This command will take you through the configuration of gcloud.

Your current configuration has been set to: [default]

You can skip diagnostics next time by using the following flag:
  gcloud init --skip-diagnostics

Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.                                                                                                                                                           
Reachability Check passed.
Network diagnostic passed (1/1 checks passed).

You must log in to continue. Would you like to log in (Y/n)?  y

Go to the following link in your browser:

    ****redacted****

Enter authorization code: ****redacted****


Updates are available for some Google Cloud CLI components.  To install them,
please run:
  $ gcloud components update

You are logged in as: [***redacted****].

Pick cloud project to use: 
****redacted****
Please enter numeric choice or text value (must exactly match list item):  3

Your current project has been set to: [****redacted****].

There is a slight difference when uploading to any other registry:

[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
[keyring.backend] Loading Google Auth
### this is different! vvvvv
[google.auth._default] Checking None for explicit credentials as part of auth process...
[google.auth._default] Checking Cloud SDK credentials as part of auth process...
[google.auth.transport.requests] Making request: POST https://oauth2.googleapis.com/token
[urllib3.connectionpool] Starting new HTTPS connection (1): oauth2.googleapis.com:443
[urllib3.connectionpool] https://oauth2.googleapis.com:443 "POST /token HTTP/1.1" 200 None
Found authentication information for all-mpm2.

I'm at a loss at what to try next.

1

There are 1 answers

0
sonder On BEST ANSWER

As @robert-g commented, the solution can be found in https://github.com/python-poetry/poetry/issues/7545

For posterity, this fixed my issue:

poetry config http-basic.my-repo oauth2accesstoken $(gcloud auth print-access-token)