How to use tox to test your code on multiple platforms

Table of Contents

Motivation

When writing Python packages you often have to assure that the packages can be used on different platforms and Python versions

e.g. the package must work under Python 3.8 on Linux as well as with Python 3.9 on Windows.

But how do you test all those supported OS and version combinations?

Enter tox

History

tox was first released in October 2018, first stable version 0.5 came out in 2010.

Today we have the 3.25 and tox 4 is in the making.

Installation

pip install tox

Setup & Usage

At a bare minimum you need a tox.ini file in your project’s root directory with the follwing content:

[tox]
envlist = py38, py39

[testenv]
deps = pytest
commands = pytest

To run tox you just type in

tox

and Bob’s your uncle

How does it work?

tox will now do the following things automagically for you

Build the package (Make an sdist distribution by invoking python setup.py sdist)
For each listed environment (in this example Python 3.8 and 3.9)
1. Create the environment with venv
2. Install the package
3. Install additional dependencies
4. Run the commands

tox will create a .tox directory in your project

More configuration

Running on multiple OS

[tox]
envlist = {win32, linux}-{py38, py39}

[testenv]
platform = win32
deps = -r ./requirements-dev.txt
commands = coverage run -m pytest --junit-xml=reports/test_report-{envname}.xml

[testenv:linux-py38]
platform = linux|linux2

[testenv:linux-py39]
platform = linux|linux2

tox has a neat syntax to create a matrix of OS and Python versions

{win32, linux}-{py38, py39} will result in the following environments

win32-py38
win32-py39
linux-py38
linux-py39

Building wheels

If you are interested in testing the creation of wheels as well you can use a plugin like

https://pypi.org/project/tox-wheel/

or call python setup.py bdist_wheel directly in the commands section

Python 3.10

[tox]
envlist = py37, py38, py39, py310

Multiple dependencies

If your tox builds rely on more dependencies you have to list them like this:

[testenv]
deps = pytest
       pyhamcrest

But to avoid duplication you should use the dependencies from your requirements file

[testenv]
deps = -r ./requirements-dev.txt

Testing Coverage

When you add coverage and pytest-cov to your requirements you can generate test coverage

[testenv]
deps = -r ./requirements-dev.txt
commands = coverage run -m pytest --junit-xml=reports/test_report-{envname}.xml

Pitfalls

WARNING: could not copy distfile to D:\.tox\distshare

drive not mounted