We have a build chain which creates an Eclipse RCP application and a series of repositories. We want to run integration tests at the completion of the build chain. So after completion of the RCP application build we added two Jenkins jobs. The first job deploys the RCP application and installs features from the repositories. The second job executes RCPTT test scripts against the deployed application. We would like to capture code coverage from these tests and display them in SonarQube. I have modified our RCPTT runner execution to add the jacoco agent and this runs and generates a jacoco-it.exec data file. If I load this data file into EclEmma in Eclipse I can view the coverage information. I have not yet figured out how to load this data into SonarQube.
Can the raw data collected by the jacoco agent be loaded directly into SonarQube (or does it have to go through another process step before loading)?
Is the raw data collected by the jacoco agent enough to get coverage information in SonarQube or do the class files of the application need to be present at the time of loading?
Our current structure for running the RCPTT tests is a multi-module structure. There is a parent pom which will execute RCPTT executions organized by features. There will be a module for each RCPTT execution. Currently I'm just testing with one module and a few scripts. I am testing by running maven from the command line.
Parent Pom
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>com.compuware.mf.topaz.product</groupId>
<artifactId>parent</artifactId>
<version>17.2.3-SNAPSHOT</version>
<packaging>pom</packaging>
<name>MFM Components : Topaz RCPTT</name>
<parent>
<groupId>com.compuware.mf</groupId>
<artifactId>parent</artifactId>
<version>0.0.11-SNAPSHOT</version>
</parent>
<!-- RCPTT Maven Plugin and RCPTT Runner are hosted in this repository -->
<pluginRepositories>
<pluginRepository>
<id>org.jacoco</id>
<name>Maven Central repository</name>
<url>http://repo1.maven.org/maven2/</url>
</pluginRepository>
<pluginRepository>
<id>rcptt-releases</id>
<name>RCPTT Maven repository</name>
<url>https://repo.eclipse.org/content/repositories/rcptt-releases/</url>
</pluginRepository>
</pluginRepositories>
<properties>
<!-- RCPTT properties -->
<rcptt-maven-version>2.1.0</rcptt-maven-version>
<rcptt-runner-version>2.2.0-M1d</rcptt-runner-version>
<testFailureIgnore>true</testFailureIgnore>
<!-- Sonar properties -->
<sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
<!-- sonar.jacoco.reportPath>${project.basedir}\..\rcptt-coverage/jacoco-it.exec</sonar.jacoco.reportPath -->
<sonar.jacoco.itReportPath>${project.basedir}\..\rcptt-coverage/jacoco-it.exec</sonar.jacoco.itReportPath>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.7.201606060606</version>
<executions>
<execution>
<id>pre-integration-test</id>
<phase>process-resources</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${project.basedir}/../rcptt-coverage/jacoco-it.exec</destFile>
<propertyName>failsafeArgline</propertyName>
</configuration>
</execution>
<!--
Ensures that the code coverage report for integration tests after
integration tests have been run.
<execution>
<id>post-integration-test</id>
<phase>process-classes</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
< Sets the path to the file which contains the execution data. >
<dataFile>${project.build.directory}/coverage-reports/jacoco-it.exec</dataFile>
< Sets the output directory for the code coverage report. >
<outputDirectory>${project.build.directory}/jacoco-it</outputDirectory>
</configuration>
</execution>
-->
</executions>
</plugin>
<plugin>
<groupId>org.eclipse.rcptt</groupId>
<artifactId>rcptt-maven-plugin</artifactId>
<version>${rcptt-maven-version}</version>
<extensions>true</extensions>
<configuration>
<!-- This element describes where to get an AUT to run tests -->
<aut>
<!-- Use AUT from previous build step. By specifying path -->
<explicit>${project.basedir}/../com.compuware.mf.topaz.rcptt.topaz-install-byant/aut/</explicit>
<injections>
<!-- features are optional - when omitted, all features from given site will be installed -->
<injection>
<site>${test-repository}</site>
<features>
<feature>${test-feature}</feature>
</features>
</injection>
</injections>
<vmArgs>
<vmArg>${failsafeArgline}</vmArg>
</vmArgs>
</aut>
<runner>
<version>${rcptt-runner-version}</version>
<vmArgs>
<vmArg>-Xmx2048m</vmArg>
</vmArgs>
</runner>
<testOptions>
<!--How many seconds Runner should wait for application startup.-->
<autStartupTimeout>600</autStartupTimeout>
<!-- Timeout for all tests, in seconds (90 minutes)-->
<execTimeout>5400</execTimeout>
<!-- Timeout for a single test case, in seconds -->
<testExecTimeout>900</testExecTimeout>
<!-- application-under-test connection timeout in seconds -->
<!-- setting this up in case the install being long running -->
<connectTimeout>600</connectTimeout>
<!-- contextWaitforjobsTimeout>120000</contextWaitforjobsTimeout> -->
<jobHangTimeout>300000</jobHangTimeout>
<!-- print memory usage -->
<memoryUsage/>
<!-- When set to true, in case of test failure AUT will be restarted. This significantly
slows down execution, but may be useful for some test suites -->
<restartAUTOnFailure>false</restartAUTOnFailure>
</testOptions>
<projects>
<project>${project.basedir}/../rcptt.workbench</project>
<project>${project.basedir}/../rcptt.hostservices</project>
</projects>
<suites>
<suite>SuiteZosConsole</suite>
</suites>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.jacoco</groupId>
<artifactId>org.jacoco.agent</artifactId>
<version>0.7.7.201606060606</version>
</dependency>
</dependencies>
<modules>
<module>../com.compuware.mf.topaz.rcptt.hostservices-test</module>
</modules>
Module Pom
<project>
<modelVersion>4.0.0</modelVersion>
<artifactId>com.compuware.mf.topaz.rcptt.hostservices-test</artifactId>
<groupId>com.compuware.mf.topaz.product</groupId>
<version>17.2.3-SNAPSHOT</version>
<parent>
<groupId>com.compuware.mf.topaz.product</groupId>
<artifactId>parent</artifactId>
<version>17.2.3-SNAPSHOT</version>
<relativePath>../com.compuware.mf.topaz.rcptt.test.parent</relativePath>
</parent>
<properties>
<test-repository>http://dtw-svngateway.prodti.compuware.com/CommonBundleDepot/products/Enterprise/hostservices/${buildType}</test-repository>
<test-feature>com.compuware.frameworks.hostservices.rcptt.test.feature.feature.group</test-feature>
</properties>
<!-- RCPTT Maven Plugin provides this packaging type -->
<build>
<plugins>
<plugin>
<groupId>org.eclipse.rcptt</groupId>
<artifactId>rcptt-maven-plugin</artifactId>
<version>${rcptt-maven-version}</version>
</plugin>
</plugins>
</build>
<packaging>rcpttTest</packaging>
The first command below executes RCPTT. The second command executes the sonar goal.
mvn --fail-at-end -DbuildType=trunk-nightly-repo clean verify
mvn -e -B sonar:sonar -Dsonar.host.url=http://localhost:9000
RCPTT Log
[INFO] Scanning for projects...
[INFO] Building MFM Components : Topaz RCPTT 17.2.3-SNAPSHOT
[INFO] ...
[INFO] --- jacoco-maven-plugin:0.7.7.201606060606:prepare-agent (pre-integration-test) @ parent ---
[INFO] failsafeArgline set to -javaagent:C:\\Users\\dresser\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.7.7.201606060606\\org.jacoco.agent-0.7.7.201606060606-runtime.jar=destfile=C:\\p\\e46\\topaz\\trunk-all\\com.compuware.mf.topaz.rcptt.test.parent\\..\\com.compuware.mf.topaz.rcptt.test.parent\\target\\jacoco-it.exec
[INFO]
[INFO] --- rcptt-maven-plugin:2.1.0:execute (default-execute) @ com.compuware.mf.topaz.rcptt.hostservices-test ---
[INFO] Runner command line is cmd.exe /X /C ""C:\Program Files\Java\jdk1.8.0_91\jre\bin\java.exe" -Xmx2048m -jar C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test\target\runner\eclipse\plugins\org.eclipse.equinox.launcher_1.3.200.v20160318-1642.jar -application org.eclipse.rcptt.runner.headless -data C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test\target\runner-workspace -aut C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test\..\com.compuware.mf.topaz.rcptt.topaz-install-byant\aut\eclipse -autWsPrefix C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test\target\aut-ws- -autConsolePrefix C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test\target\results\aut-console- -autVMArgs -javaagent:C:\\Users\\dresser\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.7.7.201606060606\\org.jacoco.agent-0.7.7.201606060606-runtime.jar=destfile=C:\\p\\e46\\topaz\\trunk-all\\com.compuware.mf.topaz.rcptt.hostservices-test\\..\\com.compuware.mf.topaz.rcptt.test.parent\\target\\jacoco-it.exec -q7report C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test\target\results\com.compuware.mf.topaz.rcptt.hostservices-test.report -htmlReport C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test\target\results\com.compuware.mf.topaz.rcptt.hostservices-test.html -junitReport C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test\target\surefire-reports\TEST-com.compuware.mf.topaz.rcptt.hostservices-test.xml -import C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test\target\projects\com.compuware.mf.topaz.rcptt.hostservices-test;C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test/../rcptt.workbench;C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test/../rcptt.hostservices -injection:site http://dtw-svngateway.prodti.compuware.com/CommonBundleDepot/products/Enterprise/hostservices/trunk-nightly-repo;com.compuware.frameworks.hostservices.rcptt.test.feature.feature.group -testOptions autStartupTimeout=600;connectTimeout=600;execTimeout=5400;jobHangTimeout=300000;memoryUsage=null;restartAUTOnFailure=false;testExecTimeout=900 -timeout 5400 -shutdownListenerPort 8347 -suites SuiteZosConsole"
[INFO] The execution timeout is set to 5400 seconds
[INFO] Started at Thu Jan 12 09:13:34 EST 2017
[INFO] RCPTT Runner version: 2.2.0
[INFO] ...
[INFO] Testcase Artifacts:7
[INFO] ...
[INFO] Pass 1 (7) processed. 0 failed. spent: 0:29, 1:27 mins remaining. CommandHistory. time: 28150ms
[INFO] Pass 2 (7) processed. 0 failed. spent: 0:59, 0:59 mins remaining. ConnectToHost. time: 29032ms
[INFO] Pass 3 (7) processed. 0 failed. spent: 1:19, 0:47 mins remaining. GetActiveSystemLog. time: 18693ms
[INFO] Pass 4 (7) processed. 0 failed. spent: 1:33, 0:31 mins remaining. IssueZOSConsoleCommand. time: 13788ms
[INFO] Pass 5 (7) processed. 0 failed. spent: 1:45, 0:15 mins remaining. OpenSystemLogs. time: 10854ms
[INFO] Pass 6 (7) processed. 0 failed. spent: 1:57, 0:00 mins remaining. SystemLogOptions. time: 11423ms
[INFO] Pass 7 (7) processed. 0 failed. spent: 2:21, 0:00 mins remaining. ZOSandTSOConsole. time: 20994ms
[INFO] Process terminated. Shut down AUTs
[INFO] Finished at Thu Jan 12 09:17:20 EST 2017
[INFO] Runner exit code is: 0
[INFO] ...
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 04:04 min
[INFO] Finished at: 2017-01-12T09:17:27-05:00
[INFO] Final Memory: 13M/160M
[INFO]
Sonar Log
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] MFM Components : Topaz RCPTT
[INFO] com.compuware.mf.topaz.rcptt.hostservices-test
[INFO] Downloading: http://dtw-mfmstbldsvr.nasa.cpwr.corp:8090/nexus/content/groups/public/org/apache/maven/plugins/maven-metadata.xml
[INFO] Downloading: https://repo.eclipse.org/content/repositories/rcptt-releases/org/apache/maven/plugins/maven-metadata.xml
[INFO] Downloading: https://repo.eclipse.org/content/repositories/rcptt-releases/org/sonarsource/scanner/maven/maven-metadata.xml
[INFO] Downloading: http://dtw-mfmstbldsvr.nasa.cpwr.corp:8090/nexus/content/groups/public/org/sonarsource/scanner/maven/maven-metadata.xml
[INFO] Downloaded: http://dtw-mfmstbldsvr.nasa.cpwr.corp:8090/nexus/content/groups/public/org/apache/maven/plugins/maven-metadata.xml (14 KB at 59.0 KB/sec)
[INFO] Downloading: http://dtw-mfmstbldsvr.nasa.cpwr.corp:8090/nexus/content/groups/public/org/codehaus/mojo/maven-metadata.xml
[INFO] Downloaded: http://dtw-mfmstbldsvr.nasa.cpwr.corp:8090/nexus/content/groups/public/org/codehaus/mojo/maven-metadata.xml (22 KB at 259.9 KB/sec)
[INFO] Downloading: https://repo.eclipse.org/content/repositories/rcptt-releases/org/codehaus/mojo/maven-metadata.xml
[INFO] Downloaded: http://dtw-mfmstbldsvr.nasa.cpwr.corp:8090/nexus/content/groups/public/org/sonarsource/scanner/maven/maven-metadata.xml (240 B at 0.5 KB/sec)
[INFO] Downloading: https://repo.eclipse.org/content/repositories/rcptt-releases/org/sonarsource/scanner/maven/sonar-maven-plugin/maven-metadata.xml
[INFO] Downloading: http://dtw-mfmstbldsvr.nasa.cpwr.corp:8090/nexus/content/groups/public/org/sonarsource/scanner/maven/sonar-maven-plugin/maven-metadata.xml
[INFO] Downloaded: http://dtw-mfmstbldsvr.nasa.cpwr.corp:8090/nexus/content/groups/public/org/sonarsource/scanner/maven/sonar-maven-plugin/maven-metadata.xml (557 B at 0.7 KB/sec)
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building MFM Components : Topaz RCPTT 17.2.3-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- sonar-maven-plugin:3.2:sonar (default-cli) @ parent ---
[INFO] User cache: C:\Users\dresser\.sonar\cache
[INFO] Load global repositories
[INFO] Load global repositories (done) | time=230ms
[INFO] User cache: C:\Users\dresser\.sonar\cache
[INFO] Load plugins index
[INFO] Load plugins index (done) | time=5ms
[INFO] SonarQube version: 5.6.4
[INFO] Default locale: "en_US", source code encoding: "windows-1252" (analysis is platform dependent)
[INFO] Process project properties
[INFO] Load project repositories
[INFO] Load project repositories (done) | time=175ms
[INFO] Load quality profiles
[INFO] Load quality profiles (done) | time=96ms
[INFO] Load active rules
[INFO] Load active rules (done) | time=1049ms
[WARNING] 'sonar.dynamicAnalysis' is deprecated since version 4.3 and should no longer be used.
[WARNING] 'sonar.dynamicAnalysis' is deprecated since version 4.3 and should no longer be used.
[INFO] Publish mode
[INFO] ------------- Scan com.compuware.mf.topaz.rcptt.hostservices-test
[INFO] Load server rules
[INFO] Load server rules (done) | time=366ms
[INFO] Base dir: C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test
[INFO] Working dir: C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.hostservices-test\target\sonar
[INFO] Source paths: pom.xml
[INFO] Source encoding: windows-1252, default locale: en_US
[INFO] Index files
[INFO] 0 files indexed
[INFO] Sensor Lines Sensor
[INFO] Sensor Lines Sensor (done) | time=0ms
[INFO] Sensor Sonar-Sonargraph-Plugin [3.5]
[INFO] ----------------------------------------------------------------
[INFO] Sonargraph: Skipping project com.compuware.mf.topaz.rcptt.hostservices-test [com.compuware.mf.topaz.product:com.compuware.mf.topaz.rcptt.hostservices-test], since no Sonargraph rules are activated in current SonarQube quality profile.
[INFO] ----------------------------------------------------------------
[INFO] Sensor Sonar-Sonargraph-Plugin [3.5] (done) | time=3ms
[INFO] Sensor SCM Sensor
[INFO] Sensor SCM Sensor (done) | time=0ms
[INFO] Sensor XmlFileSensor
[INFO] Sensor XmlFileSensor (done) | time=0ms
[INFO] Sensor Zero Coverage Sensor
[INFO] Sensor Zero Coverage Sensor (done) | time=1ms
[INFO] Sensor Code Colorizer Sensor
[INFO] Sensor Code Colorizer Sensor (done) | time=0ms
[INFO] Sensor CPD Block Indexer
[INFO] Sensor CPD Block Indexer (done) | time=0ms
[INFO] ------------- Scan MFM Components : Topaz RCPTT
[INFO] Base dir: C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.test.parent
[INFO] Working dir: C:\p\e46\topaz\trunk-all\com.compuware.mf.topaz.rcptt.test.parent\target\sonar
[INFO] Source encoding: windows-1252, default locale: en_US
[INFO] Sensor Lines Sensor
[INFO] Sensor Lines Sensor (done) | time=0ms
[INFO] Sensor Sonar-Sonargraph-Plugin [3.5]
[INFO] ----------------------------------------------------------------
[INFO] Sonargraph: Skipping project MFM Components : Topaz RCPTT [com.compuware.mf.topaz.product:parent], since no Sonargraph rules are activated in current SonarQube quality profile.
[INFO] ----------------------------------------------------------------
[INFO] Sensor Sonar-Sonargraph-Plugin [3.5] (done) | time=0ms
[INFO] Sensor SCM Sensor
[INFO] Sensor SCM Sensor (done) | time=0ms
[INFO] Sensor XmlFileSensor
[INFO] Sensor XmlFileSensor (done) | time=0ms
[INFO] Sensor Zero Coverage Sensor
[INFO] Sensor Zero Coverage Sensor (done) | time=0ms
[INFO] Sensor Code Colorizer Sensor
[INFO] Sensor Code Colorizer Sensor (done) | time=0ms
[INFO] Sensor CPD Block Indexer
[INFO] Sensor CPD Block Indexer (done) | time=0ms
[INFO] Calculating CPD for 0 files
[INFO] CPD calculation finished
[INFO] Analysis report generated in 165ms, dir size=15 KB
[INFO] Analysis reports compressed in 17ms, zip size=5 KB
[INFO] Analysis report uploaded in 44ms
[INFO] ANALYSIS SUCCESSFUL, you can browse http://localhost:9000/dashboard/index/com.compuware.mf.topaz.product:parent
[INFO] Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
[INFO] More about the report processing at http://localhost:9000/api/ce/task?id=AVmTCzfaIYaDC7fI_-4A
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] MFM Components : Topaz RCPTT ....................... SUCCESS [ 38.800 s]
[INFO] com.compuware.mf.topaz.rcptt.hostservices-test ..... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 43.343 s
[INFO] Finished at: 2017-01-12T09:18:55-05:00
[INFO] Final Memory: 20M/297M
Everything runs without error, but there is no coverage information loaded in sonar. At one point I was getting JaCoCoSensor errors that it could not find the report file. I'm not sure what I changed to make these go away, but it seems like it is not processing the same because even if I rename jacoco-exec.it the maven execution with the sonar:sonar goal runs the same.
Is the JaCoCoSensor the component that loads jacoco coverage data? If so, is there documentation on this component some where?
SonarQube 5.6.4 Java Plugin 4.4.0.8066 jacoco-maven-plugin 0.7.7.210606060606
It seems this question, Adding jacoco integration tests coverage for Sonar, solves our problem but it is high level and does not include an examples. Also, it does not answer the questions that I have above.
Based on an answer to a another post, we would need to run our regression tests in the context of our project build. The way we are doing the tests now we are collecting and trying to load coverage data in a context where the source is not present.