Inconsistent behaviour between maven-surefire and tycho-surefire, with jacoco not generating reports

Question

I'm working on creating a pom for a project and adding test cases to it. The project is an eclipse plugin.

Compiling the project with tycho works just fine, the only problem is during testing: If I run both maven-surefire-plugin tests and tycho-surefire-plugin-tests, the former performs all the tests as expected, while the latter gives the following error:

Execution test of goal org.eclipse.tycho:tycho-surefire-plugin:1.7.0:test failed: Tycho build extension not configured for MavenProject

I would be perfectly fine to just add <skipTests>true</skipTests> to the tycho-surefire-plugin while keeping maven-surefire-plugin on; the problem is even that way, jacoco refuses to create the coverage site, with the following (non error) message:

Skipping JaCoCo execution due to missing execution data file.

I tried to look for solutions of both, but any combination of the solutions I found doesn't lead me to having a working coverage site. Maven really makes me quite confused, especially with tycho around, so I'd apreciate any explanation on top of the actual fix.

Here is my pom:

<?xml version="1.0"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>
    
    <groupId>mygroupid</groupId>
    <artifactId>myartifactid</artifactId>
    <name>myname</name>
    <packaging>eclipse-test-plugin</packaging>
    
    <properties>
        <tycho-version>1.7.0</tycho-version>
    </properties>

    <parent>
        <groupId>parentgroupid</groupId>
        <artifactId>parent</artifactId>
        <version>0.9.5</version>
    </parent>
    
    <dependencies>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-api</artifactId>
            <version>5.6.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter-engine</artifactId>
            <version>5.6.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <version>0.8.5</version>
        </dependency>
        
        <dependency>
            <groupId>javassist</groupId>
            <artifactId>javassist</artifactId>
            <version>3.12.1.GA</version>
        </dependency>
    </dependencies>

    <build>
        <testSourceDirectory>src/test/java/</testSourceDirectory>
        <plugins>
            

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.12.4</version>
                
                <configuration>
                </configuration>

                <executions>
                    <execution>
                        <id>test</id>
                        <phase>test</phase>
                        <configuration>
                            <includes>
                                <include>**/Test_*.java</include>
                            </includes>
                        </configuration>
                        <goals>
                            <goal>test</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.eclipse.tycho</groupId>
                <artifactId>tycho-surefire-plugin</artifactId>
                <version>${tycho-version}</version>
                <configuration>
                    <skipTests>true</skipTests>
                </configuration>
                <executions>
                    <execution>
                        <id>test</id>
                        <phase>test</phase>
                        <configuration>
                            <includes>
                                <include>**/Test_*.java</include>
                            </includes>
                        </configuration>
                        <goals>
                            <goal>test</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            
            <plugin>
                <groupId>org.jacoco</groupId>
                <artifactId>jacoco-maven-plugin</artifactId>
                <version>0.8.5</version>
                <configuration>
                    <output>file</output>
                    <append>true</append>
                    <includes>
                        <include>**/path_to_source/**/*</include>
                    </includes>
                </configuration>
                <executions>
                    <execution>
                        <id>jacoco-initialize</id>
                        <goals>
                            <goal>prepare-agent</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>jacoco-site</id>
                        <phase>test</phase>
                        <goals>
                            <goal>report</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
         
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.5.1</version>
                <executions>
                    <execution>
                        <id>compiletests</id>
                        <phase>test-compile</phase>
                        <goals>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

And here is my parent pom:

<?xml version="1.0" encoding="UTF-8"?>
<project>
    <modelVersion>4.0.0</modelVersion>
    <groupId>parentgroupid</groupId>
    <artifactId>parent</artifactId>
    <version>0.9.5</version>
    <packaging>pom</packaging>
    <modules>
        <module>moduleid</module>
    </modules>

    <properties>
        <tycho-version>1.7.0</tycho-version>
    </properties>

    <repositories>
     <repository>
         <id>eclipse-2020-06</id>
         <layout>p2</layout>
         <url>http://download.eclipse.org/releases/2020-06</url>
     </repository>
    </repositories>

    <build>
        <plugins>
            <plugin>
                <groupId>org.eclipse.tycho</groupId>
                <artifactId>tycho-maven-plugin</artifactId>
                <version>${tycho-version}</version>
                <extensions>true</extensions>
                
                <configuration>
                    <includeAllDependencies>true</includeAllDependencies>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

Answer 1

Of course there won't be any test result for the JaCoCo due to you are using very old Surefire version 2.12.4. This version was not created for JUnit5. Use the latest version 3.0.0-M5 and see the tutorial.

If you want to have tiny POM, remove the dependency junit-jupiter-engine due to you do not need to have an access to the JUnit internals in your test code. The Surefire will download it shortly before the test runtime.

Answer 2

Your POM has several errors. Let's start with the root cause and then other priorities from high to low.

Whole problem is that Surefire does not know about JaCoCo. You have to tel "him" this way (see jacoco.agent) which "wires" both. Pls ead the documentation in the JaCoCo project:

<properties>
  <jvm.args.tests>-Xmx2048m -Xms1024m -XX:SoftRefLRUPolicyMSPerMB=50 -Djava.awt.headless=true -Djdk.net.URLClassPath.disableClassPathURLCheck=true</jvm.args.tests>
<properties>
...
      <artifactId>maven-surefire-plugin</artifactId>
      <configuration>
        <argLine>${jvm.args.tests} ${jacoco.agent}</argLine>
      </configuration>
...

The next error is with the way how you use plugins. The plugin jacoco-maven-plugin must be used only in the plugins section. The problem is that you use it also in the dependencies section. You do not want to have it on the classpath. It is job of the property jacoco.agent to put the jacoco agent on the test classpth only but there the JaCoCo plugin must start before the Surefire plugin.

The next thing i do not understand is the config of the compiler. Why you have this?

             <executions>
                <execution>
                    <id>compiletests</id>
                    <phase>test-compile</phase>
                    <goals>
                        <goal>testCompile</goal>
                    </goals>
                </execution>
            </executions>

I have second question regarding the packaging. I have never seen this one. It isn't a standard packaging.

<packaging>eclipse-test-plugin</packaging>

Has the Eclipse plugin any special binary form of the archive file?