Skip to content

billing: store Stripe customer and invoice IDs instead of matching on metadata #3102

Description

@jshearer

We never store the ID of the Stripe objects we create. Every code path that needs "the Stripe customer for this tenant" or "the Stripe invoice for this bill row" re-derives it at runtime by searching Stripe metadata:

  • Customers are found by searching metadata["estuary.dev/tenant_name"] (billing_types::customer_search_query).
  • Invoices are found by searching the compound key (customer, estuary.dev/invoice_type, estuary.dev/period_start, estuary.dev/period_end) (billing_types::InvoiceMetadata / InvoiceSearch). The doc comment on InvoiceMetadata already flags this as debt: neither internal.manual_bills nor internal.billing_historicals models a stable primary key, so identity is reconstructed from mutable fields.

This identity model is fragile in three independent ways — it's mutable (bill dates get corrected, which silently changes the identity), erasable (deleting a Stripe-side object deletes the only record it ever existed), and eventually consistent (Stripe's Search API lags writes by seconds and is heavily rate-limited).

Consequences:

  • publish-invoices re-selects a multi-month manual_bills row on every monthly run in its date range, and relies on the metadata search to find the invoice it created previously. When the match fails (dates edited, invoice created by hand in Stripe, stray draft deleted) it creates a duplicate; when it succeeds against a finalized invoice it errors instead.
  • The GraphQL StripeInvoiceLoader issues one Search call per invoice and hits 429s. billing: list invoices per customer instead of per-invoice Stripe search #3054 works around this and notes storing the invoice ID as the real fix.
  • Customer lookups race Stripe's search-index lag. This is why Customer::create needs a deterministic idempotency key, and why the live-Stripe integration test needs wait_for_customer_searchable and still flakes in CI.

Fix

Add nullable ID columns and retrieve by ID everywhere. Metadata search remains only as a one-time adoption path: on a hit, write the ID back; on a miss, create and write back.

  • tenants.stripe_customer_id
  • internal.manual_bills.stripe_invoice_id
  • internal.billing_historicals.stripe_invoice_id, exposed through invoices_ext

In publish-invoices, with a stored ID: update the invoice if it's still a draft, skip it if it's open or paid, and warn without recreating if it's voided or deleted. A manual bill then produces exactly one invoice, regardless of how many months it spans or how its dates are edited.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions