Platform Dependent Python Coverage Test with Tox

Last updated on December 20, 2020

When testing Python programs, is often used in measuring code coverage, and enforcing 100% code coverage is regarded as a good practice:

# .coveragerc
# Enforce 100% coverage test
fail_under = 100
show_missing = True

However, if there are some lines of code that are platform dependent (i.e., they are never executed on at least one platform), code coverage tests usually fail. For example, the following code snippet would always lead to a coverage that is less than 100% on a platform other than Windows:

if != 'nt':
    # Do something if the OS is not Windows...

You can ask to ignore this block by adding a comment # pragma: no cover, but then would ignore it on all platforms, including all non-Windows platforms. If you use tox for testing, this issue can be resolved cleanly.

First, mark your Python code that you would like to exclude with some platform-specific coverage comments:

if != 'nt':  # pragma: no cover windows. No execution of this branch on Windows
    # Do something if the OS is not Windows...

For Linux and MacOS exclusions, use # pragma: no cover linux and # pragma: no cover macos, respectively. We will now try to make ignore these specifically marked lines. For the example above, we will make ignore the marked if-block on Windows and only on Windows.

In tox.ini, add the following part to make sure this tox configuration is platform-aware (see the tox documentation for how this works):

# tox.ini
envlist = py{36,37,38,39}-{linux,macos,windows}
platform = linux: linux
           macos: darwin
           windows: win32
setenv =
    linux: PLATFORM = linux
    macos: PLATFORM = macos
    windows: PLATFORM = windows

Then, create a template coveragerc file, named, including the following section along with your other settings:

exclude_lines =
    pragma: no cover {platform}

These lines mean that should ignore a line if pragma: no cover {platform} is present.

Back to tox.ini, include the following lines:

# tox.ini
setenv =
    COVERAGE_RCFILE = {envtmpdir}/coveragerc
commands_pre =
    {envpython} -c 'from pathlib import Path; Path(r"{env:COVERAGE_RCFILE}").write_text(Path("").read_text().format(platform="{env:PLATFORM}"))'
commands =
    coverage run -m pytest -v

These lines read from, substitute {platform} with the appropriate platform name, and write it to a temporary file that recognizes as the configuration file for this run.

Now, running tox -e py39-windows (or other Python versions) will ignore the lines marked with # pragma: no cover windows for coverage purposes, and tox -e py39-linux (or other Python versions and OSes) will not ignore those lines.

For a live example, check out the flake8-executable code repository, specifically tox.ini,, and the Python source file with the marks.

Leave a Reply

Your email address will not be published. Required fields are marked *