I have a similar setup as described here. The only difference is that I'm building the whole project in my pipeline. This runs without problems and the code is deployed to the server without any issues. Deployer creates a new folder for the release and symlinks the "current" folder to this new release. I also make sure to run update commands for the plugins on the target server and clear the cache multiple times. Similar to this deployer recipe. This all goes off without a hitch.
Where I'm at a loss though is that the actual changes made to the project/plugins can take hours to show up. For example: when upgrading the Shopware version, the administration will still display the old version after deployment.
After further investigation I also found that the cache is still being written in the directory of the previous release (or a few releases back if multiple deployments happen in a short time).
It seems like Shopware/Symfony is resolving the symlink of the "current" folder to the actual path of that release and storing/caching this somewhere. So when a new release is made, it just keeps using the path to the old release.
What I have tried so far:
Setting the PROJECT_ROOT env variable to the "current" folder. Unfortunately this gives issues when running db migrations during deployment.
Running the system:update:finish command after deployment.
Deleting the whole cache directory after deployment.
Has anybody run into similar issues and knows how to fix this? It's obviously not feasible to wait hours for the new deployment to 'kick in' in a production environment.
See my answer to the comment by Julien B. Turns out the opcache was not clearing correctly.