I'm trying to do an end-to-end automated deployment using AWS CodePipeline, CodeBuild and CodeCommit but source stage is falling with this if I try to do a full clone using: OutputArtifactFormat = "CODEBUILD_CLONE_REF":
[Container] 2023/11/10 13:16:05.798892 Waiting for agent ping
[Container] 2023/11/10 13:16:06.799887 Waiting for DOWNLOAD_SOURCE
repository not found for primary source and source version xxxxx
The default CODE_ZIP works just fine.
I have already added codecommit:GitPull permission to CodeBuild service role and codecommit:GetRepository permission to CodePipeline service role.
Below is the related part of the code:
// CodeBuild Project
resource "aws_codebuild_project" "this" {
name = var.app_name
service_role = aws_iam_role.codebuild.arn
concurrent_build_limit = 1
environment {
compute_type = "BUILD_GENERAL1_SMALL"
image = var.build_image
image_pull_credentials_type = "SERVICE_ROLE"
privileged_mode = false
type = "ARM_CONTAINER"
}
artifacts {
type = "CODEPIPELINE"
}
source {
type = "CODEPIPELINE"
#location = var.code_commit_https_url
buildspec = file("${path.module}/buildspec.yaml")
}
}
// CodePipeline
resource "aws_codepipeline" "this" {
name = var.app_name
role_arn = aws_iam_role.codepipeline.arn
artifact_store {
location = regex("[^:]+$", var.s3_bucket_arn)
type = "S3"
encryption_key {
id = var.pipeline_key_arn
type = "KMS"
}
}
stage {
name = "Source"
action {
category = "Source"
name = "Source"
output_artifacts = ["SOURCE_ARTIFACT"]
owner = "AWS"
provider = "CodeCommit"
role_arn = var.assume_role_arn
run_order = 1
version = "1"
configuration = {
RepositoryName = var.git_repo_source
BranchName = var.git_repo_branch
PollForSourceChanges = false
OutputArtifactFormat = "CODEBUILD_CLONE_REF"
#OutputArtifactFormat = "CODE_ZIP"
}
}
}
stage {
name = "TerraformValidate"
action { .... }
}
....
}
(the var.git_repo_source is just the name of the repo, not the full https clone URL. If fails with invalid value if I use the full URL)
This AWS Example seems to be matching with what I'm doing in TF but still failing. What am I missing or doing wrong?
This is a general solution for any problems with cross-account pipelines with git submodules in CodeCommit
This is not directly an answer to the question, but will hopefully help anyone facing similar problems.
The problem:
AWS' support for submodules quite improvable.
Workaround:
Use a CodeBuild project, that will clone the repository and initialize all submodules.
Details
Create your pipeline like this
Source stage is the "native" source stage. I won't do a full clone. It's just being used to trigger the pipeline. If you don't need that, you can remove it completely.
Source2 stage will run the CodeBuild project that will clone the repo and initialize its submodules.
Implementation
CodeBuild Project for cloning a repo
The following code is fully functional CloudFormation source code. You should be able to copy/paste and run it. You will only need on deployment of the resources. No need to create a new one for each pipeline you're using.
Important
Role Policy
The role policy is a little bit more tricky. The code below will probably not work for you as copy/paste, but it might give you a lead. Variables are explained in the bottom.
Variables:
CodePipeline
The code below shows the definition of the Source2 stage. Pay attentien how the three variables for repo uri, branch name and repo access role are being conveyed.