dotnet publish with new implicit dockerfile feature using NuGet Microsoft.NET.Build.Containers fails in GitLab cicd

Question

I am trying out the new NuGet package Microsoft.NET.Build.Containers which allows building docker image with dotnet publish.

Microsoft Docs: https://learn.microsoft.com/en-us/dotnet/core/docker/publish-as-container?view=vs-2022

It works fine on local machine pushing out an image locally as it should but fails in GitLab cicd.

Project (added following to a sln file):

# create a new project and move to its directory
dotnet new mvc -n my-awesome-container-app
cd my-awesome-container-app

# add a reference to a (temporary) package that creates the container
dotnet add package Microsoft.NET.Build.Containers

# publish your project for linux-x64
dotnet publish --os linux --arch x64 -c Release -p:PublishProfile=DefaultContainer

# run your app using the new container
docker run -it --rm -p 5010:80 my-awesome-container-app:1.0.0

Cicd pipeline:

image: mcr.microsoft.com/dotnet/sdk:7.0

stages:          # List of stages for jobs, and their order of execution
  - publish

build-job:       # This job runs in the build stage, which runs first.
  stage: publish
  script:
      - dotnet publish --os linux --arch x64 --configuration Release -p:PublishProfile=DefaultContainer

The error produced (omitted repo):

MSBuild version 17.4.1+9a89d02ff for .NET
  Determining projects to restore...
  Restored /builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj (in 827 ms).
  my-awesome-container-app -> /builds/.../dotnetpublishdockerimage/my-awesome-container-app/bin/Release/net7.0/linux-x64/my-awesome-container-app.dll
  my-awesome-container-app -> /builds/.../dotnetpublishdockerimage/my-awesome-container-app/bin/Release/net7.0/linux-x64/publish/
  Building image 'my-awesome-container-app' with tags 1.0.0 on top of base image mcr.microsoft.com/dotnet/aspnet:7.0
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018: The "CreateNewImage" task failed unexpectedly. [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018: System.AggregateException: One or more errors occurred. (An error occurred trying to start process 'docker' with working directory '/builds/.../dotnetpublishdockerimage/my-awesome-container-app'. No such file or directory) [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018:  ---> System.ComponentModel.Win32Exception (2): An error occurred trying to start process 'docker' with working directory '/builds/.../dotnetpublishdockerimage/my-awesome-container-app'. No such file or directory [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018:    at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018:    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018:    at System.Diagnostics.Process.Start(ProcessStartInfo startInfo) [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018:    at Microsoft.NET.Build.Containers.LocalDocker.Load(Image x, String name, String tag, String baseName) in D:\a\_work\1\s\Microsoft.NET.Build.Containers\LocalDocker.cs:line 19 [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018:    --- End of inner exception stack trace --- [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018:    at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018:    at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018:    at System.Threading.Tasks.Task.Wait() [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018:    at Microsoft.NET.Build.Containers.Tasks.CreateNewImage.Execute() in D:\a\_work\1\s\Microsoft.NET.Build.Containers\CreateNewImage.cs:line 243 [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]
/root/.nuget/packages/microsoft.net.build.containers/0.2.7/build/Microsoft.NET.Build.Containers.targets(124,9): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [/builds/.../dotnetpublishdockerimage/my-awesome-container-app/my-awesome-container-app.csproj]

Searching for a possible solution I came across and article for it for GitHub actions and it also works there. Also the example in there I have used for this questions project.

Article: https://devblogs.microsoft.com/dotnet/announcing-builtin-container-support-for-the-dotnet-sdk/

Also the GitLab runner is using Docker executor. Perhaps it doesn't work inside a container. The article for GitHub shows ubuntu-latest but I do not know if it is running in a container underneath or not.

Anyone know what is going on and have a solution?

Answer 1

Looking further into it I found the issue and a solution as well.

There are 2 problems here.

1. In order to build images (part of dotnet publish) docker-in-docker (dind) is required.
2. The .net sdk image does not provide dind support

---

To deal with the first problem there are a couple of changes needed.

First add dnid as a service in cicd:

services:
  - docker:dind

Nex assign the docker host variable:

variables:
  DOCKER_HOST: tcp://docker:2375/

And finally to enable privileged mode on docker runner (config.toml):

[[runners]]
...
  [runners.docker]
...
    privileged = true
...

Restart of docker runner may be required.

There can also be an issue with tls. It can be ignored (although not advised for privileged mode) by adding empty certs dir to cicd:

variables:
  DOCKER_TLS_CERTDIR: ""

---

The deal with the second problem an alternative is to use the image 'docker' which provides dind support. From this there are 2 options:

1. Install the .net sdk inside:

before_script:
  - apk add dotnet7-sdk

Official docs: https://learn.microsoft.com/en-us/dotnet/core/install/linux-alpine

2. Create a new docker image based off of docker with sdk installation added and then use this image in pipeline.

---

There may be other more elegant ways to solve this but it will do as a workaround.

A sample cicd pipeline including publishing to registry:

stages: 
  - publish

variables:
  DOCKER_HOST: tcp://docker:2375/
  DOCKER_TLS_CERTDIR: ""

services:
  - docker:dind

docker_test:
  stage: publish
  image: docker
  before_script:
    - apk add dotnet7-sdk
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - dotnet publish --os linux --arch x64 --configuration Release -p:PublishProfile=DefaultContainer
    - IMAGE_ID=$(docker images --format='{{.ID}}' | head -1)
    - docker tag $IMAGE_ID $CI_REGISTRY_IMAGE/my-image:1.0.0
    - docker push $CI_REGISTRY_IMAGE/my-image:1.0.0