Error Handling and Logging Patterns in Python Services

When a production Python service fails, a swallowed exception or an unformatted stack trace on stdout is not enough. You need two things to work together: a small set of domain exceptions that mean something in the product, and structured logs that a log system can actually search. Print statements and a generic except Exception will not give you that.

This article is about error handling and logging in Python APIs, with FastAPI examples. It is not a full observability guide for LLM services. The aim is a consistent error shape for clients, and JSON logs that include the context you need without leaking secrets.

Start with a domain exception hierarchy

Do not raise bare ValueError or Exception for expected business failures. Give those failures a type and a stable error code. HTTP mapping can then live at the edge, and the rest of the service can raise something that still makes sense in a worker or a test.

class DomainError(Exception):
    """Base type for expected business failures."""

    def __init__(self, message: str, code: str = "INTERNAL_ERROR"):
        self.message = message
        self.code = code
        super().__init__(message)


class ResourceNotFoundError(DomainError):
    def __init__(self, resource_name: str, resource_id: str):
        super().__init__(
            message=f"{resource_name} {resource_id} was not found.",
            code="RESOURCE_NOT_FOUND",
        )


class InsufficientFundsError(DomainError):
    def __init__(self, available: str, required: str):
        super().__init__(
            message="This account does not have enough funds for the request.",
            code="INSUFFICIENT_FUNDS",
        )

Keep messages safe to return to a client. Do not put account internals, SQL, or vendor payloads in message. If you need those details, log them separately.

Map domain errors once, at the API layer

FastAPI lets you register an exception handler so routes do not repeat the same try/except. Unexpected errors should still become a generic 500 to the client, with the stack trace only in logs.

from fastapi import FastAPI, Request, status
from fastapi.responses import JSONResponse

app = FastAPI()


@app.exception_handler(DomainError)
async def domain_error_handler(request: Request, exc: DomainError):
    status_code = status.HTTP_400_BAD_REQUEST
    if isinstance(exc, ResourceNotFoundError):
        status_code = status.HTTP_404_NOT_FOUND
    if isinstance(exc, InsufficientFundsError):
        status_code = status.HTTP_409_CONFLICT

    return JSONResponse(
        status_code=status_code,
        content={"error": {"code": exc.code, "message": exc.message}},
    )


@app.exception_handler(Exception)
async def unhandled_error_handler(request: Request, exc: Exception):
    # Log the exception with request context, then return a generic body.
    return JSONResponse(
        status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
        content={"error": {"code": "INTERNAL_ERROR", "message": "The request could not be completed."}},
    )

A stable error body — code plus message — is easier for clients than a mix of string details and framework default pages. Validation errors from Pydantic can stay in FastAPI’s official 422 shape, or you can wrap them so every error has the same envelope. Pick one and keep it consistent.

Be careful with a catch-all Exception handler: you still want framework behaviour for things like HTTPException and request validation. Register domain handlers first, and do not swallow the exceptions FastAPI already knows how to render unless you intend to replace them.

Log structured JSON, not free-text lines

Text logs are hard to query. Structured JSON lets you filter on request_id, user_id, or an error code in CloudWatch, Datadog, Elasticsearch, or whatever you already use. structlog is one common way to do this in Python.

import structlog

structlog.configure(
    processors=[
        structlog.contextvars.merge_contextvars,
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer(),
    ]
)

logger = structlog.get_logger()

logger.info("payment_processed", amount_cents=9950, currency="USD")

Bind a request_id at the start of the request, often from an incoming X-Request-ID header or a new UUID, and include it on every log line and in the error response when you can. That is the difference between “the API failed” and “here is the path of this one call.”

Log the event and the identifiers you need to find it. Do not log access tokens, passwords, API keys, raw card data, or full personal documents. If a payload is useful for debugging, log a hash, an id, or a short redacted summary.

Habits that keep errors and logs useful

• Raise domain errors for expected failures, and let unexpected ones bubble to the generic handler.
• Return a code the client can branch on, not only a human sentence.
• Include request_id in logs and, when practical, in the error body.
• Keep stack traces in logs, never in client responses.
• Log at the point you have context, then re-raise or return. Do not log the same error in three layers.
• Use log levels honestly: info for normal events, warning for expected problems, error for failures that need attention.

If you run workers as well as an API, use the same exception types and the same log shape in both. A failed job should record the same code and request_id (or job id) that the API would have used. That is how you follow a request from the HTTP edge into a background process.

What not to do

• Do not return str(exc) from a catch-all handler. That is how SQL, vendor messages, and file paths end up in a client.
• Do not log entire request bodies by default. They often contain personal data or secrets you did not mean to store in the log system.
• Do not create a new logger configuration in every module. Configure once at startup, then get a logger by name or with structlog.get_logger().
• Do not treat a 500 as a normal way to signal a business rule. If the user can correct the request, use a domain error and a 4xx status.

If you already have metrics and traces, attach the same request_id there too. Logs, traces, and errors are much more useful when they share an identifier. You can add that later, but only if the request id exists in the first place.

The baseline is small: a domain error type, one mapping to HTTP, JSON logs, a request id, and a rule that secrets never appear in either place. Most teams already have pieces of this. The work is to make them consistent, so a failure in production can be found and understood without opening the server and reading a traceback by hand. Once that path is reliable, you can add richer tooling without changing how the service reports errors.

Summary

Custom domain exceptions make business failures explicit, and a single FastAPI handler can map them to a stable HTTP error shape. Structured JSON logging, with a request id and without secrets, makes those failures searchable later. Together they are a baseline for production Python services, whether the process is an API or a worker.

If you want help tightening error handling and logging in a Python or FastAPI service, get in touch.