Old Hat, New Tricks: Makefiles for Python Projects
Makefiles have been a staple of the C/C++ toolchain for decades. In a nutshell: GNU Make executes
discrete steps that are defined in a Makefile
. The steps defined in a Makefile
can have
dependencies, so that when a step is executed, all the dependent steps are executed first. This
enables effective version control and cleaner documentation for the manual steps that are needed to
build and run a project.
Like many GNU utilities, it’s been around for a while and is easily overlooked by developers until they encounter a project that uses them effectively. I recently had that experience in some recent Python repositories I’ve been working in. At first I was skeptical, but there are many benefits to using them in modern projects that I didn’t previously consider:
- Less documentation to write and maintain
- Dependencies between steps are built-in to the Makefile
- Just a brief summary of
Makefile
steps for your project will suffice
Previously I used a stack of bash scripts to achieve the same goal. But I found that there’s a lot
of careful organisation needed to avoid user error caused by executing steps in an incorrect order.
Now I find that with using the Makefile
, much shorter documentation is needed – as all the
details and the dependencies between steps are handled in the construction of the Makefile
.
The steps in make
can also execute any shell script, so steps can still be written in scripts and
orchestrated with make
.
One sneaky “gotcha” with writing a Makefile
is that the steps require indentation with one tab
character, not spaces (I found this
blog post, which contains an email from the author of make
who regrets this decision with good
humor!). I use this syntax file to handle this automatically when writing a Makefile
in vim:
1" ~/.vim/ftplugin/make.vim
2set noexpandtab shiftwidth=8 softtabstop=0
So why is it useful in Python? There are so many ways to manage dependencies in Python (and of
course, none of them quite meet the 100% mark). Now instead of meticulously documenting how to use
the tool of choice for a project, I simply document which tool is needed and let the Makefile
contain all the details.
For example, here’s a Makefile I would write for a Python project that uses poetry
for dependency
management, pytest
for testing, ruff
for linting and formatting, and docker
for building
images:
1SHELL := /usr/bin/env bash
2.SHELLFLAGS := -eu -o pipefail -c
3
4
5.PHONY: _check-python-version, install-uv, install-py-deps, install-py-deps-dev, check-tests, check-lint, check-format, check-all, build-image, build-image-dev, load-dev-shell
6
7
8_check-python-version:
9 @version=$$(python --version 2>&1); \
10 if [ "$$version" != "Python 3.13.7" ]; then \
11 echo "Error: Python version is not 3.12.7. Detected version: $$version"; \
12 exit 1; \
13 fi
14
15install-poetry: _check-python-version
16 @brew install pipx
17 @pipx install poetry==1.8.4 --python=$(which python)
18
19install-py-deps:
20 @poetry install --sync --only=main
21
22install-py-deps-dev:
23 @poetry install --sync --only=main,dev
24
25check-lint:
26 @poetry run pytest
27
28check-lint:
29 @poetry run ruff check
30
31check-format:
32 @poetry run ruff format --check .
33
34check-all: check-tests check-lint check-format
35
36build-image-base:
37 @docker build -t "my-container-base" -f Dockerfile.base .
38
39build-image-dev: build-image
40 @docker build -t "my-container-dev" -f Dockerfile.dev .
41
42load-dev-shell: build-dev-image
43 @docker exec -it "my-container-dev" bash
The targets in .PHONY
ensure that the steps execute every time they’re called, and @
suppresses
the command output. Using this file, all my steps are version controlled, unwieldy flags and options
for commands don’t need to be copy-pasted, and users with brew
can get up and running with a few
simple commands:
1$ make install-poetry
2$ make install-py-deps-dev
3$ make check-all
4$ make load-dev-shell
The Makefile
is here to stay. It’s simplified my development process, and it’s now one of the
first things I’ll write for a new project
References: