Code coverage with Jacoco in android while using Orchestrator

Question

I am trying to get coverage report from our test suite which needs to run with Android Test Orchestrator.

There is a fixed issue which references another fixed issue.

Following their suggestion I have added

testInstrumentationRunnerArguments coverageFilePath: '/data/local/tmp/codeCoverage'

But I'm still getting

 java.io.IOException: No coverage data to process in directories [[buildDr]/outputs/code_coverage/stageTestDebugAndroidTest/connected]

I think it might be related to this line in Test Release Note Archive page

Only enable orchestrator coverage handling if both 'coverage' and 'coverageFilePath' arguments are passed.

I have tried adding

testInstrumentationRunnerArguments coverage: 'true'

It doesn't make a difference and I couldn't find any info on what should actually be passed.

Do you know what I'm missing?

A link to a working example would also be really nice.

Answer 1

This seems to be fixed with a few new params, as shown in this sample: https://github.com/android/testing-samples/tree/main/runner/AndroidTestOrchestratorWithTestCoverageSample

I just needed to bump this library: coreVersion = "1.6.0-alpha05", besides that it worked as is.

---

We need to add the orchestrator:

androidTestUtil "androidx.test:orchestrator:" + rootProject.orchestratorVersion
androidTestUtil "androidx.test.services:test-services:" + rootProject.testServicesVersion

Enable the useTestStorageService and clearPackageData flags on the runner:

testInstrumentationRunnerArguments clearPackageData: "true"
testInstrumentationRunnerArguments useTestStorageService: "true"

Setup the flags for code coverage and orchestrator:

buildTypes {
    debug {
        testCoverageEnabled true
    }
}

testOptions {
    execution "ANDROIDX_TEST_ORCHESTRATOR"
}

And lastly, make sure you're pointing to the right *.ec files, if you're using a custom JaCoCo task, this one is not shown on the GitHub sample.

include("outputs/code_coverage/debugAndroidTest/connected/*/*.ec")

---

If that's still not working, some of the libraries and versions might need to be tweaked, these are the ones that worked for me (most of them come as is on the repository):

ext {
    androidxAnnotationVersion = "1.2.0"
    guavaVersion = "30.1.1-android"
    coreVersion = "1.6.0-alpha05"
    extJUnitVersion = "1.2.0-alpha01"
    runnerVersion = "1.6.0-alpha03"
    rulesVersion = "1.6.0-alpha01"
    testServicesVersion = "1.5.0-alpha01"
    orchestratorVersion = "1.5.0-alpha01"
    espressoVersion = "3.6.0-alpha01"
    truthVersion = "1.1.3"
}