Charging someone twice is the easiest bug to ship

I built a billing microservice as a side project: subscriptions, invoices with sequential INV-YYYYMM-XXXX numbering, Stripe integration, 39 currencies, running on ECS behind FastAPI. The part I expected to be hard was the payment flow. The part that was actually hard was everything that happens when the network misbehaves.

The retry problem

A client sends a payment request. The request goes through. The response gets lost on the way back.

The client has no way to distinguish that from a request that never arrived, so it retries. Without protection, you have now charged someone twice for one purchase, and you will find out from them rather than from your monitoring.

Every payment operation goes out with an idempotency key. Stripe stores the key with the result of the first request, so a repeat with the same key returns the original outcome instead of creating a second charge. The client can retry as many times as it likes and the customer is charged once.

This is standard practice in payment systems and almost entirely absent from side projects, which is a shame, because it takes an afternoon and it is the difference between a demo and something you could actually point at money.

Currencies are not interchangeable

The naive model of multi-currency support is a number and a three letter code. That model breaks the first time someone pays in yen.

Different currencies have different minimum chargeable amounts, and the minimum is expressed in the smallest unit of that currency, which is not the same size unit across currencies. Send Stripe an amount below the minimum and the charge fails, after you have already told the customer it worked.

CURRENCY_MINIMUMS: dict[str, int] = {
    "usd": 50, "eur": 50, "gbp": 30,
    "jpy": 50, "cad": 50, "aud": 50,
    # 39 currencies total
}

def validate_charge_amount(amount: int, currency: str) -> None:
    """Validate charge meets currency minimum."""
    currency = currency.lower()
    if currency not in CURRENCY_MINIMUMS:
        raise InvalidCurrencyError(f"Unsupported currency: {currency}")
    minimum = CURRENCY_MINIMUMS[currency]
    if amount < minimum:
        raise AmountTooSmallError(
            f"Minimum for {currency.upper()}: {minimum} (got {amount})"
        )

Every payment runs through this before it touches Stripe. The validation is boring and it prevents a category of failure that would otherwise surface as a support ticket.

Pydantic does the rest of the work at the edge. For a service where a malformed currency code can reach a payment processor, strict request validation is not optional, and getting it declaratively from the type annotations is most of why I picked FastAPI over Flask here.

You do not own the state

The subscription lifecycle looks simple written down. Created, active, upgraded or downgraded, canceled, reactivated.

The complication is that Stripe is the source of truth for most of those transitions, and your database is a cache of a system you do not control. A customer's card can fail at three in the morning. A subscription can lapse while your service is redeployed.

Two options exist. Poll Stripe on a schedule and accept that your data is stale between polls, or let Stripe tell you. I went with webhooks, so state changes arrive as they happen and there is no cron job pretending to be a synchronization strategy.

The tradeoff is that webhooks are their own problem: they arrive out of order, they arrive twice, and they arrive for events you did not think you cared about. Idempotency shows up again here, for the same reason as before.

Testing something you cannot call

You cannot hit the real Stripe API in a test suite. You also cannot write meaningful billing tests without one.

The setup that worked: an in-memory SQLite database for speed and isolation, and a mocked Stripe layer so every test is deterministic. Tests are grouped by domain rather than by file structure:

Group	Tests
Customers	7
Plans	5
Subscriptions	6
Invoices and payments	6
Currencies and health	3

Twenty-seven total, which is not a large number, but each one covers a path where being wrong costs money.

The whole thing ships through a pipeline that lints, runs the suite against a real Postgres service container, builds a container image, and deploys to Fargate.

If I were starting this again I would still begin with the idempotency key. It is the cheapest thing on this list and the only one where getting it wrong shows up on somebody's credit card statement.