Host multiple test reports with gitlab pages

Question

We have a test suite in our gitlab pipeline which produces Allure test reports. To make these results available after the test, we currently publish the results to artifacts and have an allure serve running which makes them available over a subdomain, based on the branch name.

We would like to host the test results with gitlab pages for each branch. However we can only ever host one version of a page through gitlab pages at one time. This is a problem since we want to host the test results for each branch, not only for the last executed branch. It seems like this is currently not possible without hacks.

I also found this 3 year old gitlab issue about the topic which indicates this is coming in some version of gitlab in the future.

Is there a better way to do this? Or is our best bet currently to wait until this becomes available in gitlab?

Answer 1

https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#browsing-artifacts

and one HTML file that you can view directly online when GitLab Pages is enabled > (opens in a new tab). Select artifacts in internal and private projects can only > be previewed when GitLab Pages access control is enabled.

This sound to me like you just need GitLab pages enabled and then can browser HTML files from the artifacts repository.

(correct me if a m wrong)

Answer 2

The below solution, from GitLab Pages per branch : the no-compromise hack to serve preview pages, is indeed the same kind of hack, but should be satisfying enough.

You can have these features :

• main content is exposed on $CI_PAGES_URL and the path is configurable with $CURRENT_CONTENT_PATH
• Per-branch preview content is exposed on $CI_PAGES_URL/preview, with a homepage to easily navigate to branches content
• Path to root preview folder is configurable with $EPHEMERAL_BRANCHES_PATH variable to hide preview content by obfuscation
• Generated pages are associated with environments to take advantage of auto-cleaning on branch deletion
• To avoid disturbing already existing environments, pages environment are placed under a pages folder
• If main content has not been generated in current cache, or if the cache has been deleted, an error is triggered, to avoid accidental deletion
• Deletion job can be triggered manually with any cache path as input, to clean outdated data
• Code can safely be added to an existing project pipeline without causing trouble with already existing jobs
• The workflow:rules can be deleted if you already have your own, or updated to match your flow
• The job must be named pages and the artifact must be a public folder to be deployed to GitLab Pages (or you can use the pages:publish keyword)

workflow:
  rules: # disable tag pipelines and duplicate MR pipelines
    - if: $CI_COMMIT_BRANCH

variables:
  EPHEMERAL_BRANCHES_PATH: preview # subpath to ephemeral branches content for preview, anything will work

pages:
  stage: build
  image: alpine:3.18
  cache:
    key: gitlab-pages
    paths: [public]
  before_script:
    # default available 'tree' app in alpine image does not work as intended
    - apk add tree
    # CURRENT_CONTENT_PATH is defined in rules, different between main branch and ephemeral branches
    - mkdir -p public/$CURRENT_CONTENT_PATH && ls public/$CURRENT_CONTENT_PATH/..
    - | # avoid deleting main branch content when cache has been erased
      if [ "$CI_COMMIT_BRANCH" != "$CI_DEFAULT_BRANCH" ] && [ ! -d public/$CI_DEFAULT_BRANCH ]; then
        echo -e "\e[91;1m Unable to retrieve $CI_DEFAULT_BRANCH generated files from cache ; please regenerate $CI_DEFAULT_BRANCH files first\e[0m"
        exit 1
      fi
    - rm -rf public/$CURRENT_CONTENT_PATH || true # remove last version of current branch
  script:
    - ./generate-my-html.sh --output build-docs || true # insert here your code that generates documentation
    - mv --verbose build-docs public/$CURRENT_CONTENT_PATH
    - cd public/$EPHEMERAL_BRANCHES_PATH
    - tree -d -H '.' -L 1 --noreport --charset utf-8 -T "Versions" -o index.html # generate a root HTML listing all previews for easier access
  environment:
    name: pages/$CI_COMMIT_BRANCH
    action: start
    url: $CI_PAGES_URL/$CURRENT_CONTENT_PATH
    on_stop: pages-clean-preview
  rules:
    # 'main branch' is exposed at GitLab Pages root
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      variables:
        CURRENT_CONTENT_PATH: "."
    # other (short-lived) branches generation are exposed in 'EPHEMERAL_BRANCHES_PATH/branch-name-sanitized' sub path
    - variables:
        CURRENT_CONTENT_PATH: $EPHEMERAL_BRANCHES_PATH/$CI_COMMIT_REF_SLUG
  artifacts:
    paths: [public]
    expire_in: 1h

pages-clean-preview:
  stage: build
  image: alpine:3.18
  cache:
    key: gitlab-pages
    paths: [public]
  variables:
    GIT_STRATEGY: none # git files not available after branch deletion
    FOLDER_TO_DELETE: preview/$CI_COMMIT_BRANCH # an indirection to allow arbirtraty deletion when launching this job
  script:
    - rm -rf public/$FOLDER_TO_DELETE
  environment:
    name: pages/$CI_COMMIT_BRANCH
    action: stop
  rules:
    - if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
      when: manual
      allow_failure: true

Answer 3

From https://docs.gitlab.com/ee/user/project/pages/pages_access_control.html:

"You can enable Pages access control on your project if your administrator has enabled the access control feature on your GitLab instance. When enabled, only members of your project (at least Guest) can access your website"

Answer 4

If you want to show a test report summary for each pipeline build, you can use GitLab's "Unit test reports" feature. It is slightly different to GitLab Pages, but it's easier to use, because you don't have to configure and host web pages yourself.

You only need to specify the paths to the XML files of the test results, something like this:

java:
  stage: test
  script:
    - gradle test
  artifacts:
    when: always
    reports:
      junit: build/test-results/test/**/TEST-*.xml

It will show the summary on GitLab's web page of the pipeline build results, like this:

Source and guidance:

• https://docs.gitlab.com/ee/ci/testing/unit_test_reports.html
• https://docs.gitlab.com/ee/ci/testing/unit_test_report_examples.html