How to debug app via rake test with VSCode

Question

Trying to debug a minitest test on a non-rails ruby application and I have the following config in the launch.json file:

        {
            "name": "Debug a rake task",
            "type": "Ruby",
            "request": "launch",
            "useBundler": true,
            "cwd": "${workspaceRoot}",
            "showDebuggerOutput": true,
            "pathToBundler": "/home/user/.rbenv/versions/2.5.5/bin/bundle",
            "pathToRDebugIDE": "/home/user/.rbenv/versions/2.5.5/bin/rdebug-ide",
            "program": "/home/user/.rbenv/versions/2.5.5/bin/rake",
            "args": [
                "test",
            ],
            "env": {
                "LOG_LEVEL": "debug",
                "TEST": "test_file.rb",
                "TESTOPTS": "--name=test_name",
            }
        },

Even though the breakpoints of the Rakefile are hit, neither the breakpoints of test_file.rb nor the breakpoints within the app are hit.

I suspect this might be because rake task are called as separate processes but I couldn't figure out how to handle this case. How can I make the debugger stop at the breakpoints of the app and the test files?

PS: I use byebug at the moment; however, it doesn't replace a IDE based debugger.