Bamboo detached docker not available for next task

Question

I am a trying to run my Matlab docker image in Bamboo to perform certain simulations, but I got stuck after Matlab docker task is completed since this docker image seems to be unavailable for following tasks in my Bamboo plan.

Currently I have configured the Docker run task as follows

I see that it loads the MAtlab docker image and it is able to run it, but afterwards it closes it so it is not available for the next task in my plan which tries to issue the following command

'matlab myFolder -batch mainCI'

I have also tested with below command and same results as log below

'matlab${bamboo.buildNumber} -sd myFolder'

My task after running Matlab docker looks like this

and this is the log I can share

26-Mar-2024 08:58:35    Starting task 'Matlab Docker run' of type 'com.atlassian.bamboo.plugins.bamboo-docker-plugin:task.docker.cli'
26-Mar-2024 08:58:35    Running image name (myLocalUrl/matlab:r2023aWithReport)
26-Mar-2024 08:58:35    
Beginning to execute external process for build ' testing - Main test #107 '
 ... running command line: 
/usr/bin/docker run --volume /home/bamboo/bamboo-agent-home/xml-data/build-dir/myDir:/myFolder --workdir /myFolder --detach --name matlab107 myLocalUrl/matlab:r2023aWithReport
 ... in: /home/bamboo/bamboo-agent-home/xml-data/build-dir/myDir
26-Mar-2024 08:58:35    45dee212b16c467003ef482985bfe07d8b67d80b6fef00cdc1222d123e9c24cc
26-Mar-2024 08:58:36    Finished task 'Matlab Docker run' with result: Success
26-Mar-2024 08:58:36    Starting task 'Script' of type 'com.atlassian.bamboo.plugins.scripttask:task.builder.script'
26-Mar-2024 08:58:36    
Beginning to execute external process for build ' testing - Main test #107 (myDir-107)'
 ... running command line: 
/home/bamboo/bamboo-agent-home/temp/myDir-107-ScriptBuildTask-356509690805256537.sh -batch mainCI
 ... in: /home/bamboo/bamboo-agent-home/xml-data/build-dir/myDir/myFolder
26-Mar-2024 08:58:36    /home/bamboo/bamboo-agent-home/temp/myDir-107-ScriptBuildTask-356509690805256537.sh: 1: matlab -sd myFolder -batch mainCI: not found
26-Mar-2024 08:58:36    Failing task since return code of [/home/bamboo/bamboo-agent-home/temp/myDir-107-ScriptBuildTask-356509690805256537.sh -batch mainCI] was 127 while expected 0

I have been reading this Bamboo ticket and this How to avoid Docker container deletion after build completion but I haven't managed to make it work...

Any suggestion of what else can I do to make the Matlab docker available for next task?

Thanks in advance!

Answer 1

Set up your job to run in docker, not in the agent;

Then the container is kept alive for all tasks in the job. A couple of caveat's;

1. Docker runs as root in bamboo, so created files could cause cleanup issues. You can find a workaround here; https://confluence.atlassian.com/bamboo0903/configuring-the-docker-task-in-bamboo-1236445454.html under the "Permission denied on files created within a Docker container" section.
2. Bamboo doesn't manage docker instances. Every time the build agent pulls a docker image, this image is kept on the agent storage after the build finishes. We just set up a clear storage job for each agent that runs weekly but you could add also add a task to do this.
3. If your docker registry requires authorization, you'll need to make sure all build agents are pre-authorized. You can do this by setting up a script task for each agent that runs docker login.We set up a seperate build plan and just put it on a timer to run. Ideally you would use variables for your username and password so your script would just look like this; docker login -u ${bamboo.DockerUsername} -p ${bamboo.DockerPassword} id -u

For elastic agents, you can add it to the startup script by editing the image configuration.