How to structure your Python project

This is not a “my way or the highway” kind of advice. Just a bunch of good practices. A starting point if you are new to Python. You will likely inspect and adapt your own project structure over time.

Table of Contents

The bare minimum

This is a setup I recommend when just want to start with a small project and might not know by now which direction it will take:

my_project/
|-- my_project/ 
    |-- __init__.py
    |-- my_project.py
|-- tests/
    |-- __init__.py
    |-- test_my_project.py
|-- .gitignore
|-- README.md
|-- requirements.txt
|-- requirements_to_freeze.txt

The my_project package

This useful to get the source code out of the root directory, to keep it away from all the helper files.

The tests package

It is important to put your tests into a tests package (not folder, hence the __init__.py) so that all imports from your main package can be resolved!

I recommend to use pytest. See also: pytest Tutorial

The .gitignore

If you use git as your version control it is recommended that you exclude files via a .gitignore.

You should put all generated and IDE-related files into the gitignore e.g.

__pycache__/
/.idea
/venv/
/.pytest_cache/
/build
/dist

You can get a more extensive Python gitignore from

https://github.com/github/gitignore/blob/main/Python.gitignore

Requirements files

In requirements_to_freeze.txt you just put the top level dependencies, requirements.txt will contain all dependencies with exact versions. (Lock file)

Python pip and virtualenv

README

You can use markdown (README.md) or re-structured text (README.rst).

Both of them have their advantages and disadvantages.

For starters I recommend markdown.

More documentation can be put into a separate docs folder

Open Source

If you want to publish your code on github, add a file named LICENSE to the root folder and insert the licence text.

CI/CD

Travis

If you want to use travisCI you need a .travis.yml in your root directory.

How to add Travis CI to your github project

Jenkins

Another way to add CI/CD is a Jenkinsfile

Anatomy of a Jenkinsfile

Python Package

If you want to distribute your project as a Python packages on PyPI you can add a setup.py with the minimal content:

import setuptools

setuptools.setup(
    name='example_pkg',
    version='0.0.1',
    packages=['example_pkg'],
    author='Joern Boegeholz',
    author_email='boegeholz.joern@gmail.com',
    description='An example Package',
)

You can read more about packaging: Distributing your own package on PyPi

PBR

When using pbr you need the additional files

pyproject.toml
setup.cfg

in the root directory

What is python pbr?

Full blown structure

my_project/
|-- docs
|-- my_project/ 
    |-- __init__.py
    |-- my_project.py
|-- tests/
    |-- __init__.py
    |-- test_my_project.py
|-- .gitignore
|-- .travis.yml
|-- LICENSE
|-- pyproject.toml
|-- README.md
|-- requirements.txt
|-- requirements_to_freeze.txt
|-- setup.cfg
|-- setup.py

I’ve created a github template for quicker setup: https://github.com/jboegeholz/python-repo-template