Different build configurations for test schemes

Question

Is there a way to run the unit tests in a debug build and performance tests in a release build without manually selecting and running individual schemes?

I have a unit test and a performance test scheme. In the test configuration for the unit test scheme, I selected debug build, and for the performance test scheme I selected release build. If I run each scheme individually, I get a debug build and a release build respectively.

If I create another scheme that runs both of these schemes, then that new scheme will have its own build configuration. If I set a build configuration of debug for this new scheme, then I will get a debug build for my performance tests as well.

Answer 1

You could use different test bundles and include/exclude what you want.

What I mean is creating your own custom test targets (bundles) and using the Xcode Test Navigator.

So for example creating a MyUnitBundleTests target and a MyPerformanceBundleTests target. They would be two separate test 'bundles' where you choose to included/exclude classes, methods etc.

The Test Navigator displays a hierarchical list of the test bundles and associated, classes, and methods etc included in a project.

You can enable and disable test bundles, classes, and methods selectively by Control-clicking the items in the test navigator list and choosing Enable or Disable from the shortcut menu, thereby enabling or disabling the items in the scheme.

Testing with Xcode - Quick Start

Testing with Xcode - Running Tests and Viewing Results

There are several additional interactive ways to run tests. Xcode runs tests based on what test targets are included and enabled in a scheme. The test navigator allows you to directly control which test targets, classes, and methods are included, enabled, or disabled in a scheme without having to use the scheme editor.

Not sure if this is what you're wanting but this is the only option I can think of for testing without going up to the Scheme Editor.

Answer 2

There might be a way to achieve the desired behaviour but it is more of a hack.

The main problem I see is the desired build configuration for the test action of the scheme. Since the build configuration applies to all targets in the scheme you need to find a way to trick Xcode into building your target with both Debug and Release configuration. That being said, here is the idea:

1. Export the release configuration for your target into an xcconfig file (xcodebuild -scheme "schemeName" -showBuildSettings >> release.xcconfig and make sure it only contains the release settings)
2. Create another target for your performance tests that builds the same source files
3. Set the debug configuration of this new target to the aforementioned xcconfig file of the release configuration (that is basically the part where we fool Xcode)
4. Add the original and the newly created target to the build action of your scheme and add the test bundles (assuming they reside in a separate test bundle)
5. Select the debug build configuration for the test action.

If you test your scheme it should now build the sources in both debug and release and should run your unit tests on the debug build configuration and the performance tests on the release configuration.

The setup is a bit fragile as you need to add new source files to both targets. You should be able to automate the process using a ruby script and the xcodeproj gem though.

Hope that helps.