Poetry fails to publish package to Jfrog's PyPi artifactory via Trusted Publishing

Question

I'm having a trouble to publish packages to Jfrog's PyPi artifactory via Trusted Publishing. I've tried already multiple methods and all results in the same 401 error Wrong username was used. No clue what that could mean since I'm not using any username to authenticate with Jfrog. I'm missing here something. I checked and the process of fetching the access token from Jfrog works - GitHub OIDC provider is able to fetch the access token from Jfrog. It seems to me that the access token is somehow wrongly interpreted and used by poetry to authenticate with Jfrog.

  packages_publish:
    needs:
      - generate_packages
      - packages_test
    runs-on: ubuntu-20.04
    container: <PYTHONRUNTIME_IMAGE>
    env:
      OIDC_AUDIENCE: 'jfrog-github'
      OIDC_ITEGRATION_NAME: 'github-oidc-integration'
    #if: github.ref == 'refs/heads/main'
    permissions: write-all
    steps:
      - name: Install dependencies for authorization
        run: apt update && apt install -y jq
      - name: Get ID Token
        id: idtoken
        run: |
          ID_TOKEN=$(curl -sLS -H "User-Agent: actions/oidc-client" -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
          "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=$OIDC_AUDIENCE" | jq .value | tr -d '"')
          echo "ID_TOKEN=${ID_TOKEN}" >> $GITHUB_OUTPUT
      - name: Fetch Access Token from Artifactory
        id: fetch_access_token
        env:
          ID_TOKEN: ${{ steps.idtoken.outputs.id_token }}
        run: |
          ACCESS_TOKEN=$(curl \
          -X POST \
          -H "Content-type: application/json" \
          https://example.jfrog.io/access/api/v1/oidc/token \
          -d \
          "{\"grant_type\": \"urn:ietf:params:oauth:grant-type:token-exchange\", \"subject_token_type\":\"urn:ietf:params:oauth:token-type:id_token\", \"subject_token\": \"$ID_TOKEN\", \"provider_name\": \"$OIDC_ITEGRATION_NAME\"}" | jq .access_token | tr -d '"')
          echo ACCESS_TOKEN=$ACCESS_TOKEN >> $GITHUB_OUTPUT
      - uses: actions/checkout@v3
      - name: Publish to artifactory
        env:
          POETRY_PYPI_TOKEN_EXAMPLE: ${{ steps.fetch_access_token.outputs.access_token }}
          POETRY_REPOSITORIES_EXAMPLE_URL: 'https://example.jfrog.io/artifactory/api/pypi/pypi-general-local'
          ACCESS_TOKEN: '${{ steps.fetch_access_token.outputs.access_token }}'
        run: |
          cd packages/<package-example>
          sed -i "0,/\(version = \"[0-9]\+.[0-9]\+\)\"/s//\1.${{ github.run_number }}\"/" pyproject.toml
          poetry config pypi-token.example $POETRY_PYPI_TOKEN_EXAMPLE
          poetry publish --build -r example -vvv

Full error Publish output:

Publishing package-example (0.1.50) to example
 - Uploading package-example-0.1.50-py3-none-any.whl 0%
 - Uploading package-example-0.1.50-py3-none-any.whl 100%
  Stack trace:
  1  /opt/poetry/venv/lib/python3.10/site-packages/poetry/publishing/uploader.py:265 in _upload_file
      263│                     bar.display()
      264│                 else:
    → 265│                     resp.raise_for_status()
      266│             except (requests.ConnectionError, requests.HTTPError) as e:
      267│                 if self._io.output.is_decorated():
  HTTPError
  401 Client Error:  for url: https://example.jfrog.io/artifactory/api/pypi/pypi-general-local
  at /opt/poetry/venv/lib/python3.10/site-packages/requests/models.py:1021 in raise_for_status
      1017│                 f"{self.status_code} Server Error: {reason} for url: {self.url}"
      1018│             )
      1019│ 
      1020│         if http_error_msg:
    → 1021│             raise HTTPError(http_error_msg, response=self)
      1022│ 
      1023│     def close(self):
      1024│         """Releases the connection back to the pool. Once this method has been
      1025│         called the underlying ``raw`` object must not be accessed again.
The following error occurred when trying to handle this error:
  Stack trace:
  11  /opt/poetry/venv/lib/python3.10/site-packages/cleo/application.py:327 in run
       325│ 
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a
  10  /opt/poetry/venv/lib/python3.10/site-packages/poetry/console/application.py:190 in _run
       188│         self._load_plugins(io)
       189│ 
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│ 
   9  /opt/poetry/venv/lib/python3.10/site-packages/cleo/application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│ 
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│ 
   8  /opt/poetry/venv/lib/python3.10/site-packages/cleo/application.py:473 in _run_command
       471│ 
       472│         if error is not None:
     → 473│             raise error
       474│ 
       475│         return terminate_event.exit_code
   7  /opt/poetry/venv/lib/python3.10/site-packages/cleo/application.py:457 in _run_command
       455│ 
       456│             if command_event.command_should_run():
     → 457│                 exit_code = command.run(io)
       458│             else:
       459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED
   6  /opt/poetry/venv/lib/python3.10/site-packages/cleo/commands/base_command.py:117 in run
       115│         io.input.validate()
       116│ 
     → 117│         return self.execute(io) or 0
       118│ 
       119│     def merge_application_definition(self, merge_args: bool = True) -> None:
   5  /opt/poetry/venv/lib/python3.10/site-packages/cleo/commands/command.py:61 in execute
        59│ 
        60│         try:
     →  61│             return self.handle()
        62│         except KeyboardInterrupt:
        63│             return 1
   4  /opt/poetry/venv/lib/python3.10/site-packages/poetry/console/commands/publish.py:82 in handle
        80│         )
        81│ 
     →  82│         publisher.publish(
        83│             self.option("repository"),
        84│             self.option("username"),
   3  /opt/poetry/venv/lib/python3.10/site-packages/poetry/publishing/publisher.py:86 in publish
        84│         )
        85│ 
     →  86│         self._uploader.upload(
        87│             url,
        88│             cert=resolved_cert,
   2  /opt/poetry/venv/lib/python3.10/site-packages/poetry/publishing/uploader.py:107 in upload
       105│ 
       106│         try:
     → 107│             self._upload(session, url, dry_run, skip_existing)
       108│         finally:
       109│             session.close()
   1  /opt/poetry/venv/lib/python3.10/site-packages/poetry/publishing/uploader.py:191 in _upload
       189│     ) -> None:
       190│         for file in self.files:
     → 191│             self._upload_file(session, url, file, dry_run, skip_existing)
       192│ 
       193│     def _upload_file(
  UploadError
  HTTP Error 401:  | b'{\n  "errors" : [ {\n    "status" : 401,\n    "message" : "Wrong username was used"\n  } ]\n}'
  at /opt/poetry/venv/lib/python3.10/site-packages/poetry/publishing/uploader.py:271 in _upload_file
      267│                 if self._io.output.is_decorated():
      268│                     self._io.overwrite(
      269│                         f" - Uploading {file.name} FAILED"
      270│                     )
    → 271│                 raise UploadError(e)
      272│             finally:
      273│                 self._io.write_line("")
      274│ 
      275│     def _register(self, session: requests.Session, url: str) -> requests.Response:
Error: Process completed with exit code 1.

It works with poetry's http-basic.example authentication method. The POETRY_HTTP_BASIC_EXAMPLE_USERNAME must be the username associated with the fetched access-token from Jfrog (user that was set in identity method within Jfrog's OIDC Integration and which permissions allow him to push to desired artifactory).

- name: Publish to artifactory
  env:
    POETRY_HTTP_BASIC_EXAMPLE_USERNAME: '${{ vars.<var_name>' }}
    POETRY_HTTP_BASIC_EXAMPLE_PASSWORD: '${{ steps.fetch_access_token.outputs.access_token }}'
    POETRY_REPOSITORIES_EXAMPLE_URL: 'https://example.jfrog.io/artifactory/api/pypi/pypi-general-local'
  run: |
    cd packages/<package-example>
    sed -i "0,/\(version = \"[0-9]\+.[0-9]\+\)\"/s//\1.${{ github.run_number }}\"/" pyproject.toml
    poetry config http-basic.example $POETRY_HTTP_BASIC_EXAMPLE_USERNAME $POETRY_HTTP_BASIC_EXAMPLE_PASSWORD
    poetry publish --build -r example -vvv

I'd rather use only the access-token without having to specify the username for the associated fetched access-token. From my understanding this should work with the pypi-token.example method, but it fails with the 401 error: Wrong username was used.

Answer 1

I figured out how to make it work using basic authentication and Jfrog's OpenID Connect Integration.

The reason why it doesn't work with pypi-token.example is because JFrog access tokens do not work the same way as PyPI tokens.

With this method, you don't need to store poetry username and password credentials in GitHub secrets.

Token (JWT) will be used as the password and the username that is associated with the token will be parsed from the token.

To make this work, you need to:

1. Configure an OIDC Integration on Jfrog
2. Add Identity Mappings for the already created OIDC Integration - Identity Mappings defines a set of conditions for matching an incoming OIDC claim to a specific authorization scope. If an incoming OIDC claim matches the defined conditions, Identity Mappings will generate a short-lived access token to grant access for publishing.

Example:

packages_publish:
    runs-on: ubuntu-latest
    container: ghcr.io/<poetry_image>
    env:
      OIDC_AUDIENCE: 'jfrog-github' # must match the exact name set in Jfrog's OIDC Integration
      OIDC_ITEGRATION_NAME: 'github-oidc-integration' # must match the exact name set in Jfrog's OIDC Integration
    permissions:
      id-token: write # this permission is mandatory for Jfrogs's OpenID Connect Integration/trusted publishing
      contents: read # required if job’s steps are manipulating with the repository, e.g. actions/checkout@v4.
      packages: read # required if job is using a container image
    steps:
      - name: Install dependencies for auth
        run: apt update && apt install -y jq
      - name: Get ID Token
        run: |
          ID_TOKEN=$(curl -sLS -H "User-Agent: actions/oidc-client" -H "Authorization: Bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
          "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=$OIDC_AUDIENCE" | jq .value | tr -d '"')
          echo "ID_TOKEN=${ID_TOKEN}" >> $GITHUB_ENV
      - name: Fetch Access Token from Artifactory
        env:
          ID_TOKEN: ${{ env.ID_TOKEN }}
        run: |
          ACCESS_TOKEN=$(curl \
          -X POST \
          -H "Content-type: application/json" \
          https://example.jfrog.io/access/api/v1/oidc/token \
          -d \
          "{\"grant_type\": \"urn:ietf:params:oauth:grant-type:token-exchange\", \"subject_token_type\":\"urn:ietf:params:oauth:token-type:id_token\", \"subject_token\": \"$ID_TOKEN\", \"provider_name\": \"$OIDC_ITEGRATION_NAME\"}" | jq .access_token | tr -d '"')
          echo "::add-mask::$ACCESS_TOKEN"
          echo "ACCESS_TOKEN=${ACCESS_TOKEN}" >> $GITHUB_ENV
      - name: Parse username from access-token
        env:
          ACCESS_TOKEN: ${{ env.ACCESS_TOKEN }}
        shell: bash
        run: |
          USERNAME=$(jq -R 'split(".") | .[1] | @base64d | fromjson' <<< "$ACCESS_TOKEN" | jq -r .sub | sed 's/.*users\///')
          echo USERNAME=${USERNAME} >> $GITHUB_ENV
      - uses: actions/checkout@v4
      - name: Publish to artifactory
        env:
          JFROG_SERVICE_USERNAME: ${{ env.USERNAME }}
          JFROG_SERVICE_ACCESS_TOKEN: ${{ env.ACCESS_TOKEN }}
        run: |
          poetry config repositories.example"https://example.jfrog.io/artifactory/api/pypi/example"
          poetry config http-basic.example$JFROG_SERVICE_USERNAME $JFROG_SERVICE_ACCESS_TOKEN
          poetry publish --build -r example