python.
python6 min read

Pydantic Settings v2: Layered Env Config Without the Boilerplate

Build env-driven config in Python with Pydantic Settings v2 — nested models, layered .env overrides, validation, and secret types that survive logs.

Pydantic Settings v2: Layered Env Config Without the Boilerplate

Most Python services accumulate config the same way. A few os.environ.get() calls, a hand-rolled bool parser, a forgotten default that bites in staging, and eventually a 200-line config.py nobody trusts. Pydantic Settings v2 collapses that pile into a typed model with validation, layered sources, and secret handling \u2014 and it does it in roughly the same number of lines you would have written for the worst-case getenv chain.

This piece is opinionated: when you need anything beyond two flat env vars, reach for pydantic-settings over python-dotenv + manual parsing. The break-even point is about five settings.

Why split pydantic-settings from Pydantic core

Pydantic v2 moved settings management out of the main pydantic package. You install it separately:

pip install pydantic-settings
# or, in a uv workspace
uv add pydantic-settings

The split matters because BaseSettings is no longer a free import \u2014 older tutorials show from pydantic import BaseSettings, which silently fails on v2. The new import is from pydantic_settings import BaseSettings, SettingsConfigDict. If you copy a snippet that errors with ImportError: cannot import name 'BaseSettings' from 'pydantic', you are reading a v1 example.

The benefit of the split is that core Pydantic stays serialization-focused, while settings get features that only make sense for env loading: source priority, secret directories, CLI parsing, and .env file layering.

A minimal config that already earns its keep

from pydantic import Field
from pydantic_settings import BaseSettings, SettingsConfigDict


class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=".env",
        env_file_encoding="utf-8",
        env_prefix="APP_",
        case_sensitive=False,
        extra="ignore",
    )

    debug: bool = False
    port: int = Field(default=8000, ge=1, le=65535)
    database_url: str
    log_level: str = "INFO"


settings = Settings()

A few details that pay off later:

  • env_prefix="APP_" namespaces every variable. APP_PORT=9000 populates port. This prevents collisions with PATH, HOME, or anything CI injects.
  • port: int = Field(..., ge=1, le=65535) validates the range at startup. A typo like APP_PORT=80000 raises a ValidationError before your server tries to bind.
  • extra="ignore" tolerates unrelated env vars. Set it to "forbid" in tests to catch typos like APP_DATBASE_URL.

Compare against the manual version: int(os.getenv("APP_PORT", "8000")) happily accepts "80000", and os.getenv("APP_DEBUG", "false").lower() == "true" quietly returns False when someone sets APP_DEBUG=1. Pydantic Settings parses 1, true, yes, on correctly because it uses Pydantic's coercion layer.

Nested models: stop flattening config

Most services accumulate config in clusters: database settings, Redis settings, observability settings. Flat env vars force you to flatten the model too, which loses cohesion. Pydantic Settings handles nested submodels via a delimiter:

from pydantic import BaseModel, Field
from pydantic_settings import BaseSettings, SettingsConfigDict


class DatabaseConfig(BaseModel):
    url: str
    pool_size: int = 5
    pool_timeout: float = 30.0


class RedisConfig(BaseModel):
    url: str = "redis://localhost:6379/0"
    max_connections: int = 10


class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=".env",
        env_nested_delimiter="__",
        env_prefix="APP_",
    )

    debug: bool = False
    database: DatabaseConfig
    redis: RedisConfig = RedisConfig()

Now APP_DATABASE__URL=postgres://... and APP_DATABASE__POOL_SIZE=20 populate the nested model. The double-underscore is a convention because single-underscore would be ambiguous (APP_DATABASE_URL could mean either database_url or database.url).

Why prefer nested over flat:

  1. The model documents itself. A new engineer reads Settings and knows there are exactly two infrastructure dependencies.
  2. Submodels are reusable. DatabaseConfig works in tests with DatabaseConfig(url="sqlite:///:memory:") without touching env vars.
  3. Validation localizes. A bad pool_size reports as database.pool_size in the error, not buried in a flat namespace.

The trade-off: env var names get longer. If you have a small flat config (say, three values), the nested layout is over-engineering. The break-even is roughly six settings or two clusters.

Layered .env overrides

Real deployments need layered defaults: a checked-in .env.example, a developer-local .env, and a per-environment .env.production. Pydantic Settings supports a list:

class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=(".env.example", ".env", ".env.local"),
        env_file_encoding="utf-8",
    )

Order matters: later files override earlier ones, but actual environment variables always win over any .env file. This is the priority chain Pydantic Settings uses by default, from lowest to highest:

  1. Field defaults in the model
  2. .env files (in declared order)
  3. Environment variables
  4. Secrets directory
  5. Init arguments (Settings(database_url="..."))

You can override the priority by subclassing settings_customise_sources if you have unusual needs (e.g. a JSON config file should beat env vars). For 90% of services, the default chain is what you want.

A pattern that works well in container deploys: ship .env.example in the repo with safe placeholders, mount .env.production from a secret manager at /etc/yourapp/.env.production, and never rely on the shell environment alone. This way, a developer running locally and a Kubernetes pod load config the same way, with the same precedence rules.

Secret types that survive logging

pydantic.SecretStr and SecretBytes are the most underused features in the v2 ecosystem. They render as ********** in repr(), str(), and JSON serialization, and only expose the underlying value through .get_secret_value():

from pydantic import SecretStr
from pydantic_settings import BaseSettings


class Settings(BaseSettings):
    api_key: SecretStr
    database_password: SecretStr


settings = Settings()
print(settings)
# api_key=SecretStr('**********') database_password=SecretStr('**********')

# Use the real value only when calling out
import httpx
httpx.get("https://api.example.com", headers={"Authorization": f"Bearer {settings.api_key.get_secret_value()}"})

Why this matters: structured logging libraries like structlog, loguru, and even print(settings) in a debugger will dump the entire object. With str fields, your secrets land in CloudWatch the first time someone debugs a startup issue. With SecretStr, they don't.

The cost is one extra .get_secret_value() call at the consumption site. That is a fair trade for not having a credential leak in your logs at 3 a.m.

For file-mounted secrets (Kubernetes / Docker secrets), use secrets_dir:

class Settings(BaseSettings):
    model_config = SettingsConfigDict(secrets_dir="/run/secrets")

    database_password: SecretStr

Now /run/secrets/database_password is read as the value, and you can rotate it without restarting the container if you re-instantiate Settings.

Validation hooks for cross-field rules

Field-level validation handles individual values, but real config has cross-field rules: "if auth_mode='oauth' then oauth_client_id must be set." Use model_validator:

from typing import Literal
from pydantic import model_validator
from pydantic_settings import BaseSettings


class Settings(BaseSettings):
    auth_mode: Literal["none", "basic", "oauth"] = "none"
    oauth_client_id: str | None = None
    oauth_client_secret: SecretStr | None = None

    @model_validator(mode="after")
    def check_oauth_config(self) -> "Settings":
        if self.auth_mode == "oauth":
            if not self.oauth_client_id or not self.oauth_client_secret:
                raise ValueError("oauth_client_id and oauth_client_secret required when auth_mode='oauth'")
        return self

The check runs at instantiation. A misconfigured production deploy fails to boot rather than silently running with broken auth \u2014 which is what you want, because crash-on-startup is debuggable in 30 seconds and "auth doesn't work" is debuggable in 30 minutes.

Testing strategies

Two patterns that scale better than monkey-patching os.environ:

Pattern 1 \u2014 instantiate with overrides:

def test_with_test_db():
    settings = Settings(database=DatabaseConfig(url="sqlite:///:memory:"))
    assert settings.database.url.startswith("sqlite")

Pattern 2 \u2014 use a fixture that swaps the singleton:

import pytest

@pytest.fixture
def test_settings(monkeypatch, tmp_path):
    env_file = tmp_path / ".env"
    env_file.write_text("APP_DATABASE__URL=sqlite:///:memory:\
")
    monkeypatch.chdir(tmp_path)
    return Settings()

Avoid the temptation to mutate a module-level settings object. Pydantic models are validated at construction; mutating fields bypasses validation and can create states the model would have rejected.

When not to use it

Pydantic Settings is overkill for:

  • Scripts with two or three env vars \u2014 os.environ["DB_URL"] is fine
  • Tools where startup time matters in milliseconds \u2014 Pydantic v2 is fast (Rust-backed core), but importing it costs ~30ms cold
  • Configs that are mostly file-based YAML/TOML with env overrides \u2014 use dynaconf instead, which prioritizes file sources

For services with more than five settings, layered environments, secrets, or any nested structure, it pays for itself in the first config bug it catches.

References