AWS CodeBuild invoked from CodePipeline produces artefact which cannot be used for AWS Lambda

Question

I would like to automate deployment of AWS Lambda developed in java. For this I created CodePipeline which is triggered on git push command to CodeCommit repository. Next step in CodePipeline is CodeBuild project. CodeBuild uses following buildspec.yml file:

version: 0.1

phases:
  build:
    commands:
      - echo Entering build phase...
      - echo Build started on `date`
      - mvn package shade:shade
      - mv target/Output-1.0.jar .
artifacts:
  files:
    - Output-1.0.jar

When CodeBuild project is run manually it will upload jar file to s3 bucket. This jar file can be without any problem used to update lambda and everything works as expected. But if CodeBuild is run via CodePipeline, result is jar file wrapped inside zip. Since this zip cannot be used for updating lambda function, I am not sure what I should do here since CodePipeline overwrites any packaging set for CodeBuild project.

Idea is that CodePipeline triggers CodeBuild which produces output which additional lambda will took and update lambda function with it. Is it somehow possible that output of CodeBuild which is invoked from CodePipeline be jar instead of zip ? If not, what should I do here then ?

Any help is appreciated.

Answer 1

A zip or a jar file can both be used to update a Lambda Function, you just need to add a "Deploy Step" using Cloudformation to your CodePipeline.

This is a nodejs build/pipeline, try to adapt to your java project:

Project Files

buildspec.yml

version: 0.2

phases:
  install:
    commands:
      - echo install phase
  pre_build:
    commands:
      - echo pre_build phase
  build:
    commands:
      - npm install --production      
  post_build:
    commands:
      - echo post build
artifacts:
  type: zip
  files:
    - index.js      
    - node_modules/**/*
    - package.json
    - template.yml
    - configuration.json    
  discard-paths: no

configuration.json

{
  "Parameters": {
    "BucketName" : { "Fn::GetArtifactAtt" : ["Build", "BucketName"]},
    "ObjectKey" : { "Fn::GetArtifactAtt" : ["Build", "ObjectKey"]}
  }
}

template.yml (you need to add a AWS::Lambda::Permission)

AWSTemplateFormatVersion: "2010-09-09"
Description: "My Lambda Template"
Parameters:
  BucketName:
    Type: String
  ObjectKey:
    Type: String
  Roles:
    Type: String
    Default: Roles
  LambdaRole:
    Type: String
    Default: LambdaRole

Resources:

  MyLambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Description: 'My Lambda Handler'
      Handler: index.handler
      Runtime: nodejs6.10
      Timeout: 5
      Code:
        S3Bucket: 
          Ref: BucketName
        S3Key: 
          Ref: ObjectKey
      Role:
        Fn::Join:
          - ""
          - - "arn:aws:iam::"
            - !Ref AWS::AccountId
            - ":role/"          
            - Fn::ImportValue:
                Fn::Join:
                  - ""
                  - - Ref: Roles
                    - "-"
                    - Ref: LambdaRole

Roles Template

AWSTemplateFormatVersion: '2010-09-09'
Description: 'The AWS Resource Roles'
Resources:
  CodeBuildRole:    
    Type: AWS::IAM::Role    
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          Effect: Allow
          Principal:
            Service: codebuild.amazonaws.com
          Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AWSCodeBuildAdminAccess
        - arn:aws:iam::aws:policy/CloudWatchFullAccess
        - arn:aws:iam::aws:policy/AWSCodeCommitFullAccess
        - arn:aws:iam::aws:policy/AmazonS3FullAccess

  CodePipelineRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          Effect: Allow
          Principal:
            Service: codepipeline.amazonaws.com
          Action: sts:AssumeRole      
      Policies:
        -
          PolicyName: CloudFormationFullAccess
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action: 
                  - "cloudformation:*"                  
                Resource: "*"  
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AWSCodePipelineFullAccess
        - arn:aws:iam::aws:policy/AWSCodeCommitFullAccess
        - arn:aws:iam::aws:policy/AmazonS3FullAccess
        - arn:aws:iam::aws:policy/AWSCodeBuildAdminAccess        
        - arn:aws:iam::aws:policy/AWSLambdaFullAccess

  CloudFormationRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          Effect: Allow
          Principal:
            Service: cloudformation.amazonaws.com
          Action: sts:AssumeRole    
      Policies:
        -
          PolicyName: CloudFormationFullAccess
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action: "cloudformation:*"
                Resource: "*"  
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AWSCodePipelineFullAccess
        - arn:aws:iam::aws:policy/AWSCodeCommitFullAccess
        - arn:aws:iam::aws:policy/AmazonS3FullAccess
        - arn:aws:iam::aws:policy/AWSCodeBuildAdminAccess
        - arn:aws:iam::aws:policy/AWSLambdaFullAccess
        - arn:aws:iam::aws:policy/AmazonAPIGatewayAdministrator        

  LambdaRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          Effect: Allow
          Principal:
            Service: lambda.amazonaws.com
          Action: sts:AssumeRole          
      Policies:
        -
          PolicyName: CloudFormationFullAccess
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              -
                Effect: "Allow"
                Action: "cloudformation:*"
                Resource: "*"    
      ManagedPolicyArns:        
        - arn:aws:iam::aws:policy/AWSLambdaFullAccess
        - arn:aws:iam::aws:policy/AWSCodePipelineFullAccess
        - arn:aws:iam::aws:policy/AmazonSESFullAccess

Outputs:
  CodeBuildRoleOutput:
    Description: 'Maybe API CodeBuildRole ARN'
    Value: !Ref 'CodeBuildRole'
    Export: 
      Name: !Sub '${AWS::StackName}-CodeBuildRole'
  CodePipelineRoleOutput:
    Description: 'Maybe API CodePipelineRole ARN'
    Value: !Ref 'CodePipelineRole'
    Export: 
      Name: !Sub '${AWS::StackName}-CodePipelineRole'    
  CloudFormationRoleOutput:
    Description: 'Maybe API CloudFormationRole ARN'
    Value: !Ref 'CloudFormationRole'
    Export: 
      Name: !Sub '${AWS::StackName}-CloudFormationRole'
  LambdaRoleOutput:
    Description: 'Maybe API LambdaRole ARN'
    Value: !Ref 'LambdaRole'
    Export: 
      Name: !Sub '${AWS::StackName}-LambdaRole'

CodePipeline Bucket

AWSTemplateFormatVersion: '2010-09-09'
Description: 'The AWS S3 CodePipeline Bucket'
Resources:

  CodePipelineBucket:    
    Type: AWS::S3::Bucket 
    DeletionPolicy: Retain
    Properties:
      BucketName: my-code-pipeline-bucket
      VersioningConfiguration:
        Status: Enabled
      AccessControl: BucketOwnerFullControl           

Outputs:
  CodePipelineBucketOutput:
    Description: 'CodePipeline Bucket Ref'
    Value: !Ref CodePipelineBucket
    Export: 
      Name: !Sub '${AWS::StackName}-CodePipelineBucketRef'    

CodeBuild Template

AWSTemplateFormatVersion: '2010-09-09'
Description: 'Nodejs CodeBuild Template'
Parameters:
  Artifact:
    Type: String
    Default: artifact
  Roles:
    Type: String
    Default: Roles
  CodeBuildRole:
    Type: String
    Default: CodeBuildRole  

Resources:  

  NodejsCodeBuild:
    Type: AWS::CodeBuild::Project
    DeletionPolicy: Retain
    Properties:
      ServiceRole:
        Fn::ImportValue:
          Fn::Join:
            - ""
            - - Ref: Roles
              - "-"
              - Ref: CodeBuildRole    
      Artifacts: 
        Type: no_artifacts      
      Environment:
        ComputeType: BUILD_GENERAL1_SMALL
        Image: aws/codebuild/eb-nodejs-6.10.0-amazonlinux-64:4.0.0
        Type: LINUX_CONTAINER
      Source:
        Type: S3
        Location: !Ref Artifact
Outputs:
  NodejsCodeBuildOutput:
    Description: 'Nodejs CodeBuild Ref'
    Value: !Ref 'NodejsCodeBuild'
    Export: 
      Name: !Sub '${AWS::StackName}-NodejsCodeBuildRef'

CodePipeline Template

AWSTemplateFormatVersion: '2010-09-09'
Description: 'CodePipeline for Nodejs Applications'

Parameters:

  Roles:
    Type: String
    Default: Roles
  CodePipelineRole:
    Type: String
    Default: CodePipelineRole
  CloudFormationRole:
    Type: String
    Default: CloudFormationRole  
  CodePipelineBucket:
    Type: String
    Default: CodePipelineBucket
  CodePipelineBucketRef:
    Type: String
    Default: CodePipelineBucketRef
  PipelineName:
    Type: String
    Default: PipelineName
  CodeBuildProject:
    Type: String
    Default: NodejsCodeBuild
  CodeBuildProjectRef:
    Type: String
    Default: NodejsCodeBuildRef
  Branch:
    Type: String
    Default: master
  Repository:
    Type: String
    Default: my-repository-name
  LambdaStack:
    Type: String
    Default: LambdaStack


Resources:

  NodejsCodePipeline:    
    Type: AWS::CodePipeline::Pipeline
    Properties:
      Name: !Ref PipelineName
      RoleArn:
        Fn::Join:
          - ""
          - - "arn:aws:iam::"
            - !Ref AWS::AccountId
            - ":role/"          
            - Fn::ImportValue:
                Fn::Join:
                  - ""
                  - - Ref: Roles
                    - "-"
                    - Ref: CodePipelineRole


      ArtifactStore:
        Location:          
          Fn::Join:
            - ""
            - - Fn::ImportValue:
                  Fn::Join:
                    - ""
                    - - Ref: CodePipelineBucket
                      - "-"
                      - Ref: CodePipelineBucketRef
        Type: S3

      Stages:

        - Name: Source
          Actions:
            - InputArtifacts: []
              Name: Source
              ActionTypeId:
                Category: Source
                Owner: AWS
                Version: 1
                Provider: CodeCommit
              OutputArtifacts:
                - Name: Master
              Configuration:
                BranchName: !Ref Branch
                RepositoryName: !Ref Repository
              RunOrder: 1

        - Name: Build
          Actions:            
            - Name: Build                
              ActionTypeId:
                Category: Build
                Owner: AWS
                Version: 1
                Provider: CodeBuild
              InputArtifacts:
                - Name: Master
              OutputArtifacts:
                - Name: Build            
              Configuration:
                ProjectName:
                  Fn::Join:
                    - ""
                    - - Fn::ImportValue:
                          Fn::Join:
                            - ""
                            - - Ref: CodeBuildProject
                              - "-"
                              - Ref: CodeBuildProjectRef
              RunOrder: 1

        - Name: Stage
          Actions:            
            - Name: Sandbox                
              ActionTypeId:
                Category: Deploy
                Owner: AWS
                Version: 1
                Provider: CloudFormation
              InputArtifacts:
                - Name: Build
              OutputArtifacts:
                - Name: Deploy            
              Configuration:
                StackName: !Ref LambdaStack
                ActionMode: CREATE_UPDATE
                Capabilities: CAPABILITY_IAM
                TemplateConfiguration: Build::configuration.json
                TemplatePath: Build::template.yml
                ParameterOverrides: |
                  {
                    "BucketName" : { "Fn::GetArtifactAtt" : ["Build", "BucketName"]},
                    "ObjectKey" : { "Fn::GetArtifactAtt" : ["Build", "ObjectKey"]} 
                  }                
                RoleArn:
                  Fn::Join:
                    - ""
                    - - "arn:aws:iam::"
                      - !Ref AWS::AccountId
                      - ":role/"          
                      - Fn::ImportValue:
                          Fn::Join:
                            - ""
                            - - Ref: Roles
                              - "-"
                              - Ref: CloudFormationRole
              RunOrder: 1