GitLab-CI Pipeline Not Recognizing Azure CLI and Terraform in Same Job

Question

I am a 2/3 week old newbie to GitLab and have run into a GitLab pipeline issue which has left me tearing my hair out, as despite hours of research online, including the official GitLab website, I simply can't seem to find a resolution.

Basically, my pipeline is made up of a number of stages, with each stage requiring one or more jobs to be executed. I've called the first pipeline stage "prep" and it features two jobs - gitlab_job1 and gitlab_job2.

The first job, gitlab_job1 does a simple terraform version check and confirms the fact that the terraform command is recognised within that job and as per my expectation, the entire pipeline.

The second job, gitlab_job2 pulls down a container image of Azure CLI and installs it successfully. However, within this same job, I'm required to run a number of terraform commands (and possibly other scripts as well) and this is where my issue lies, because suddenly the pipeline job no longer appears to recognise any Terraform command. I'm assuming that referencing the Azure CLI image in that job has wiped out Terraform from the gitlab_job2's scope entirely, but I need it to recognise it here.

Now, I don't mind having to create a third job in this same "prep" stage within which I can then reference my Terraform image (like I did in gitlab_job1) and run the required commands, but the key issue here is that this 3rd job will also be required to recognise or have within its scope, the installed Azure CLI from gitlab_job2.

Azure CLI and Terraform have therefore got to co-exist within the same job or if kept in discrete jobs, each must be visible from the other job. What am I doing wrong? Is it even also possible to reference more than one container image in a GitLab pipeline job? I've even tried the Needs and Dependencies keywords, but all to no avail.

Below is a snippet of my .gitlab-ci.yml file, including some descriptive annotation to provide better clarity. Could greatly do with some help on this.

default:
  image: 
    name: hashicorp/terraform:1.1.7
    entrypoint: [""]

stages:
  - prep
  - build
  - deploy  

gitlab_job1:
  stage: prep
  inherit:
    default: true
  script:
    - echo "This job successfully confirms the inherited Terraform version as 1.1.7" 
    - terraform --version

gitlab_job2:
   stage: prep
   inherit:
    default: true
   image: mcr.microsoft.com/dotnet/core/sdk:3.1

   script:
     - echo "This job will install the Azure cli."
     - curl -sL https://aka.ms/InstallAzureCLIDeb | bash
     - az --version   # Verify cli version after the install

     # This next command below should then run a basic Terraform validation, but throws the error directly below it.

     - terraform --validate 
     # Error msg displayed: - '/bin/bash line 139':' terraform':' command not found.

     # Other terraform commands follow the above validation command, but also fail as a result of the same error as above.

Answer 1

In the job gitlab_job2 you have specified image: mcr.microsoft.com/dotnet/core/sdk:3.1 as the image for the job environment. This image does not contain terraform therefore, the command will not be found unless you first install terraform in the job.

gitlab_job1 works because it is using the hashicorp/terraform:1.1.7 image which does include terraform.

Consider using the terraform image and installing azure cli in it instead for gitlab_job2: -- or use an image with azure CLI and install terraform.

Working example:

gitlab_job2:
   stage: prep
   # use Azure CLI image as a starting point
   image: mcr.microsoft.com/azure-cli

   script:
     - az --version   # CLI is included in image

     # install Terraform
     - curl -o ./terraform-bin.zip https://releases.hashicorp.com/terraform/1.1.8/terraform_1.1.8_linux_amd64.zip
     - unzip ./terraform-bin.zip -d /usr/local/bin
     - rm ./terraform-bin.zip
     
     - terraform version # terraform is now installed!
     #... now you can use azure cli and terraform together.