Leverage unit testing and coverage

Leverage unit testing and coverage

• JaCoCo
• Use the "ratchet" pattern to fail the build when coverage drops. Robert Greiner talks more on this in Continuous Code Improvement Using Ratcheting This follows the agile "Boy Scout" principle
• Fluent assertions — lots of options in this area
  • AssertJ — solid choice
  • Built assertions from Junit makes is difficult for developers to distinguish "actual" values from "expected" values. This is a limitation from Java as it lacks named parameters. Other frameworks compatible with JUnit provide more fluent assertions such as AssertJ. Different choices make sense depending on your source language

Unit testing and code coverage are foundations for code quality. Your build should help you with these as much as possible. 100% coverage may seem absurd; however, levels of coverage like this come with unexpected benefits such as finding dead code in your project or helping refactoring to be simple. An example: with high coverage (say 95%+, your experience will vary) simplifying your covered code may lower your coverage as uncovered code becomes more prominent in the total ratio.

Setup for needed plugins:

• For Gradle use the java plugin
• For Maven, use more recent versions of the Maven Surefire Plugin

To see the coverage report (on passed or failed coverage), open:

• For Gradle, build/reports/jacoco/test/html/index.html
• For Maven, target/site/jacoco/index.html

This project also provides the coverage report as part of Maven's project report.

The coverage script is helpful for checking your current coverage state: try ./coverage -f all. Current limitations:

• Maven builds only
• Single module builds only

Local tweaks to coverage

It is common to be working on new code, or refactoring existing code, and see coverage drop temporarily. Sometimes you want to work on production code before working on unit tests to try out new ideas, etc. In this situation, failing the build for low coverage gets in your way.

This project uses properties for picking coverage values so you can get a green build before you are ready to make a commit or push commits to share with others.

Gradle (a decimal from 0.00 to 1.00 representing percentage):

• coverageBranches
• coverageInstructions
• coverageLines

Maven (a percentage from 0 to 100):

• coverage.branches
• coverage.instructions
• coverage.lines

An example with Maven:

# Completely disable coverage minimums temporarily
$ ./mvnw clean verify -Dcoverage.branches=0 -Dcoverage.instructions=0 -Dcoverage.lines=0

Showing coverage in a badge

One nice feature of systems like GitHub or GitLab is showing project badges in the README.md, ie, the front page when users look at a project. There is no standard badge for showing coverage, but there are several good options. The example for this project is from Vincent A. Cicirello: jacoco-badge-generator.

Tips

• With Maven, do use the available BOM (bill of materials) for JUnit. An example pom.xml block is:

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.junit</groupId>
                <artifactId>junit-bom</artifactId>
                <version>${junit.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

  This helps avoid dependency conflicts from other dependencies or plugins
• See discussion on Lombok how to sparingly leverage the @Generated annotation for marking code that JaCoCo should ignore
• Discuss with your team the concept of a "coverage ratchet". This means, once a baseline coverage percentage is agreed to, the build configuration will only raise this value, not lower it. This is fairly simple to do by periodically examining the JaCoCo report, and raising the build coverage percentage over time to match improvements in the report
• Unfortunately neither Gradle's nor Maven's JaCoCo plugin will fail your build when coverage rises! This would be helpful for supporting the coverage ratchet
• You may find mocking helpful for injection. The Java community is not of one mind on mocking, so use your judgment:
  • Mockito is the "standard" choice, and is a dependency for the sample projects. For modern versions of Mockito, please use the mockito-core dependency rather than mockito-inline. See TheFooTest.shouldRedAlertAsStaticMock for an example. Note that this project has updated to Mockito 5. See v5.0.0 release notes when updating to Mockito 5 from 4.
  • PowerMock provides additional features; however, Mockito normally covers use cases.
  • Other Modern JVM languages — these languages may prefer different mocking libraries, eg, MockK for Kotlin
  • You might consider complementary libraries to Mockito for specific circumstances, eg, System Lambda for checking STDOUT and STDERR, program exits, and use of system properties (eg, validate logging), also a dependency for the sample projects.

Note

These are generally not parallelizable tests as they alter the state of the JVM. Another is the JUnit Pioneer extension pack. If you need these, be cautious about using parallel testing features, and avoiding Flaky Tests).

• To open the report for JaCoCo, build locally and use the <project root>/build/reports/jacoco/test/html/ path (pick index.html or index.htm depending on your platform and preference).