Xcode5 Code Coverage (from cmd-line for CI builds)

Question

How can I generate code coverage with Xcode 5 and iOS7?

Prior to upgrading I was getting code coverage just fine. Now I can't see any *.gcda files being produced.

The cmd-line that I'm using is:

xcodebuild -workspace ${module.name}.xcworkspace test -scheme ${module.name} -destination OS=${module.sdk.version},name=iPad -configuration Debug

Works with AppCode

• When I execute the tests via AppCode I can see *.gcda files being produced in ~/Library/Caches/appCode20/DerivedData. . . I need this to work for my Continuous Integration builds.

Works from Xcode IDE

• Also works from Xcode IDE. . . is there a cmd-line that will produce coverage, or is this an Xcode bug?

Answer 1

Update: New accepted answer

In some cases the coverage flushing needs to be done from within the app itself. The solution's outline in this question provide details.

Answer 2

With the information from here I was able to craft this version which is the least invasive I could think of. Just add to your unit tests and run the tests as normal. The ZZZ ensures it is the last run suite of tests.

I had to ensure I added the GCC_GENERATE_TEST_COVERAGE_FILES and GCC_GENERATE_TEST_COVERAGE_FILES compiler flags to my test unit target too to get the coverage out.

//
//  Created by Michael May
//

#import <SenTestingKit/SenTestingKit.h>

@interface ZZZCodeCoverageFixForUnitTests : SenTestCase

@end

@implementation ZZZCodeCoverageFixForUnitTests

// This must run last

extern void __gcov_flush();

-(void)testThatIsntReallyATest
{
    NSLog(@"FLUSHING GCOV FILES");

    __gcov_flush();
}


@end

Edit, or another approach by Jasper:

I stripped the VATestObserver from the other answer down to this:

@interface VATestObserver : SenTestLog
@end

@implementation VATestObserver

extern void __gcov_flush(void);

- (void)applicationWillTerminate:(UIApplication*)application
{
   __gcov_flush();
  [super applicationWillTerminate:application];
}

@end

Answer 3

Some more documentation here:

https://code.google.com/p/coverstory/wiki/UsingCoverstory

and some source code to use:

https://code.google.com/p/google-toolbox-for-mac/source/browse/#svn%2Ftrunk%2FUnitTesting

You need GTMCodeCoverageApp.h/.m and GTMCodeCoverageTestsXC.h/.m or GTMCodeCoverageTestsST.h/.m depending on if you are using XCTest or SenTest.

Answer 4

(This is not the answer, but a work-around . . .I'm still very much interested in a better solution)

Use iOS 6.1 Simulator

If you're targeting iOS 6.1 or earlier as a deployment target, you can use the 6.1 simulator.

• Install the iOS6.1 Simulator via preferences/downloads

• Use the following cmd-line:

  xcodebuild -workspace ${module.name}.xcworkspace test -scheme ${module.name} -destination OS=6.1,name=iPad -configuration Debug

Answer 5

We found that we had to add a bit of code to get the gcda files to flush from the system.

Code addition is to add extern void __gcov_flush(); to the top of your file and then call __gcov_flush(); just before the entire test suite exits.

Full explanation is here: http://www.bubblefoundry.com/blog/2013/09/generating-ios-code-coverage-reports/

Answer 6

The following is a fix for SenTestKit - simply add this class to your Tests target. Something similar should be possible to do with XCTest

@interface VATestObserver : SenTestLog

@end

static id mainSuite = nil;

@implementation VATestObserver

+ (void)initialize {
    [[NSUserDefaults standardUserDefaults] setValue:@"VATestObserver" forKey:SenTestObserverClassKey];

    [super initialize];
}

+ (void)testSuiteDidStart:(NSNotification*)notification {
    [super testSuiteDidStart:notification];

    SenTestSuiteRun* suite = notification.object;

    if (mainSuite == nil) {
        mainSuite = suite;
    }
}

+ (void)testSuiteDidStop:(NSNotification*)notification {
    [super testSuiteDidStop:notification];

    SenTestSuiteRun* suite = notification.object;

    if (mainSuite == suite) {
        UIApplication* application = [UIApplication sharedApplication];
        [application.delegate applicationWillTerminate:application];
    }
}

and add

extern void __gcov_flush(void);

- (void)applicationWillTerminate:(UIApplication*)application {
    __gcov_flush();
}

Why is this working?

Tests and the tested application are compiled separately. Tests are actually injected into the running application, so the __gcov_flush() must be called inside the application not inside the tests.

The little magic with the observer only enables us to check when the tests are going to end and we trigger __gcov_flush() to be called inside the app.