ProbableOdyssey

TIL: How to parse config from env vars in Python

In an effort to collate all my runtime options and parameters into a single source of truth, I initially created a src/globals.py file with the values and their defaults:

1import os
2
3FOO: str = os.get("FOO", "my_value")
4BAR: int = int(os.get("BAR", "55"))
5BAZ: bool = os.get("BAZ", "False").lower().startswith("t")

A simple solution, these symbols can then be imported by using from src.globals import FOO, ..., but this solution has a couple of drawbacks:

For example, let’s say we have a function floob that uses a dict containing foo, bar, and baz:

1from src.globals import FOO, BAR, BAZ
2
3def floob(data: dict[str, str | int | bool] | None = None):
4    data = data or {}
5    foo = data.get("foo", FOO)
6    bar = data.get("bar", BAR)
7    baz = data.get("baz", BAZ)
8    ...

Admittedly this is a contrived example, but it can start to get unwieldy as the number of get statements increases and becomes more scattered about. Not the mention there is the possibility of a bug in globals.py when casting items to their correct types.

I came across pydantic_settings as a solution to this by providing a class to configure this.

 1from pydantic import Field
 2from pydantic_settings import BaseSettings
 3
 4
 5# An env prefix can optionally be specified as well
 6class MyConfig(BaseSettings, env_prefix="APP_"):
 7    foo: str = Field(
 8        description="foo",
 9        default="my_value",
10    )
11    bar: int = Field(
12        description="bar",
13        default=55,
14    )
15    baz: bool = Field(
16        description="baz",
17        default=False,
18    )

When this class is instantiated, the values are looked up from the given kwargs, then dynamically looked up from the environment, and then falls back to the defaults:

1# $ export APP_FOO="env_value"
2config = Config(baz=True)
3
4print(config.foo)
5# "env_value"
6print(config.bar)
7# 55
8print(config.baz)
9# True

Where this really shines is in the integration with the rest of pydantic, such as value validation:

 1from typing import Annotated
 2
 3from pydantic import Field, AfterValidator
 4from pydantic_settings import BaseSettings
 5
 6
 7def is_allowed(value: str) -> str:
 8    if value is in ["not_allowed", "wumbo"]:
 9        raise ValueError(f"Invalid string value: '{value}'")
10    return value
11
12CheckedStr = Annotated[str, AfterValidator(is_allowed)]
13
14# An env prefix can optionally be specified as well
15class MyConfig(BaseSettings, env_prefix="APP_"):
16    foo: Checked = Field(
17        description="foo",
18        default="my_value",
19    )
20    bar: int = Field(
21        description="bar",
22        default=55,
23    )
24    baz: bool = Field(
25        description="baz",
26        default=False,
27    )
28
29# $ export APP_FOO=wumbo
30_ = Config()  # Raises a validation error

The end result is a more flexible config class, cleaner definition of defaults, robust parsing of env vars without the standard boilerplate, on top of the validation and type-checking from pydantic. …

Reply to this post by email ↪