Token counts are an infrastructure measurement. Customers, however, pay for a useful result. A durable billing model has to connect the two without exposing users to the randomness of model internals.
Meter the raw event
Record provider, model, input tokens, output tokens, cached tokens, latency and request purpose at one controlled boundary. Preserve the raw facts even if the customer-facing pricing model changes later.
Price the product unit
Credits can make variable inference cost understandable, but only when a credit maps to a meaningful action. Set allowances from observed usage distributions and include retries, evaluations and failed generations in margin planning.
Design for change
Model prices and routing strategies change. Version cost calculations, keep model choice behind the product boundary and make reconciliation possible. Billing should be explainable from an immutable usage ledger.
Architecture becomes useful when its trade-offs are visible to the product—not just to the engineering team.