Picture of Stefan Lieser
Stefan Lieser

Mutation testing and continuous integration

The Clean Code Developer Initiative names the following Principles also various Practicesthat a team should apply. These include

Together, these four practices make a very important contribution to the value of correctness. The integration and unit test practices ensure automated tests. Coverage analysis ensures that the test coverage is sufficiently high. Finally, the continuous integration process ensures that tests and coverage analysis are executed fully automatically across the entire code of the project. The following section uses a small example project in C# to show how coverage analysis and continuous integration can be set up in concrete terms. This includes Stryker Mutator and GitLab are used as tools. 

Mutation testing for coverage analysis

I have already published an article about Stryker Mutator as a tool for coverage analysis in dotnetpro (see https://www.dotnetpro.de/tools/testing/mutanten-kommen-killt-2402511.html). The idea behind it is simple:

  • Change the code in one place (e.g. ">" to ">=" or "+" to "-")
  • Run all tests
  • If no test goes red, a gap has been found in the test coverage

Stryker Mutator is available for JavaScript, C# and Scala. Suitable tools can also be found for other languages.

Continuous Integration

Stryker Mutator zur Coverage Analyse

I implemented the fourth module, Continuous Integration, using GitLab as an example. I stored a sample project in C# there and then configured the CI process. In principle, the process is simple: in the root directory of the GitLab project, you create the file .gitlab-ci.yml to. This is responsible for the configuration of the CI pipeline. In detail, however, it can be a little time-consuming to gather all the details and adjust them until everything runs as desired. It is important that you always execute the commands locally on your own computer first. As long as the command does not work locally, you should not enter it in the CI. There you wait longer for the result, so that you can make faster progress locally.

The CI process runs in GitLab in Docker containers. You can either use GitLab's own infrastructure and receive 400 minutes of CI/CD time in the free plan. A runner for executing the CI pipeline can also be installed on your own machine with very little effort. Of course, a cloud server can be used for this. I got it up and running in minutes with an IONOS Cloud Server. Instructions for Ubuntu can be found here, for example: https://lindevs.com/install-gitlab-runner-on-ubuntu/

Clean code training

Closed company courses

We conduct all seminars as closed company courses for you.

If you are interested or have any questions please contact us.

execute dotnet-stryker

My example project should use the command dotnet-stryker execute. As the tool is not part of the standard scope of the Docker image dotnet/sdk, it must first be installed. The path must then be exported so that the program can be found. The following listing shows the complete CI configuration of the example. I will certainly continue working on it after this article is published.

					image: mcr.microsoft.com/dotnet/sdk:latest
    - build
    - test

    stage: build
        - dotnet --version
        - dotnet build

    stage: test
        - dotnet tool install -g dotnet-stryker
        - export PATH="$PATH:/root/.dotnet/tools"
        - cd mutation.tests
        - dotnet-stryker --break-at 95
        paths: [mutation.tests/StrykerOutput]
        untracked: false
        expire_in: 2 days

The entire project is available in a public repository at GitLab:


Line 1 defines which Docker image is to be used for the stages. In this case, it is the official .NET Core 6.0 SDK image (with "latest" currently 6.0.3 is pulled). The SDK is required so that the compiler and build system are available.

The following lines 2-4 define the two stages "build" and "test". This also determines the order in which they are executed. The definitions of the stages then follow.

The "build" stage first shows which .NET version is being used (line 9). This could be helpful later when troubleshooting. The "dotnet build" command is then executed to compile the project. Ultimately, this simply ensures that the tests are only executed when the project can be built. In real CI pipelines, artifacts would be created and stored here, e.g. NuGet packages or Docker images.

The "test" stage from line 12 initially installs dotnet-stryker as a tool. So that the command can be found later, the path is entered with export is appended (line16). The "cd" command then changes to the directory in which the test project is located. Finally, dotnet-stryker is executed there. The parameter "-break-at 95" forces the build step to fail if less than 95% of the mutations survive.

Finally, the report that dotnet-stryker creates is stored as an artifact so that it can be downloaded if required. This step can certainly be configured even more elegantly so that the report, an HTML file, can be viewed directly in GitLab.


The combination of automated tests, coverage analysis and CI is a powerful tool for ensuring correctness. A configuration is currently still missing to integrate the TestContainers into the GitLab CI process. We will stay tuned...

Our seminars

Clean Code Developer Basics

Principles and tests - The seminar is aimed at software developers who are just starting to deal with the topic of software quality. The most important principles and practices of the Clean Code Developer Initiative are taught.

to the seminar "
Clean Code Developer Trainer

Conducting seminars as a trainer - This seminar is aimed at software developers who would like to pass on their knowledge of Clean Code Developer principles and practices or Flow Design to others as a trainer.

to the seminar "
Clean Code Developer CoWorking

Online CoWorking incl. coaching -
We are often asked what developers can do to keep up with the topic of clean code development. Our answer: Meet up with other Clean Code Developers online on a regular weekly basis.

to the seminar "

Leave a Comment

Your email address will not be published. Required fields are marked *