Continuous testing with IntelliJ Idea

Question

I'm looking for an IntelliJ IDEA plugin that would run my tests each time I change my code. I looked for such a solution and I found:

• Infinitest, which works, but is inconvenient because I need to add the facet to each module, and it opens a new tool window for each module (which means 15 tool windows for me).
• Fireworks - didn't work for me, maybe it just doesn't work with IDEA 14 (in its repo I can see that last changes were made in 2009). IntelliJ also reports that it throws exceptions.

There are lots of ways I could run all my tests (including writing a simple script for this), but I'm looking for a tool that would be smart enough to rerun failed tests first, and that would understand module dependencies (so after a change in some module it would run only tests of dependent modules).

I prefer free options, but if there's something paid for a reasonable price, I would accept it as well.

Answer 1

If you are OK running tests which cover a single method chosen by you, you can try this plugin (it is continuos in the sense that you make changes to a method, then click on the method and the plugin will run all the unit tests automatically which cover that method): https://plugins.jetbrains.com/plugin/15063-unit-test-coverage-history-runner

Answer 2

IntelliJ now actually has a Toggle auto-testin the run dialog. Just run a run-configuration and then select Toggle auto-testin the run dialog.

It's not as intelligent as you would have hoped. It just reruns when it detects changes.

Answer 3

You can use the Intellij Teamcity plugin. Teamcity is a paid product but there is a free version which gives you 20 projects and 3 agents for free .

1. It has a remote run feature using which you can send in unchecked code to run tests before committing.
2. It also has options to run failed tests first

Usage instructions for Remote Run

Answer 4

I found out a way to enable SBT style auto test execution in IntelliJ studio.

We need to do two things to enable auto test execution.

1. Enabling auto project build - This can be done in settings by going into File -> Settings -> Build, Execution... -> Compiler and selecting "Build Project automatically".

2. Enabling "Toggle auto-test" in run dialog box

This will start auto testing. Although this works fine, it takes time to build the project even when my project is tiny so for larger projects it will certainly take very long time to complete the build and execute tests.

Reference: Original article which explains these steps