Budget an LLM workload with observable inputs
By Mihai
Separate token rates from request shape, retries and the rest of the bill.
A price per token is only one input
Two applications using the same model can have very different bills. One may send a short prompt and receive a short answer; another may resend a long history and several retrieved documents. Record average input tokens, average output tokens and requests per month from a representative workload. Prefer provider usage records or its tokenizer to estimates based on English word counts.
Use the two-rate formula
Multiply input tokens by the input rate and output tokens by the output rate, both stated per million tokens. Divide by one million, then multiply by request volume. At illustrative rates of $1 input and $4 output, a 2,000-input-token and 500-output-token request costs $0.004. Ten thousand such requests cost $40. The example demonstrates arithmetic and is not a current quote for any named model.
Account for retries and hidden prompt growth
An application that retries every failed operation sends more traffic than its count of successful user actions suggests. Tool loops and conversation growth also change token counts. Build scenarios around short, typical and long requests rather than applying a single optimistic average. If the provider separately bills cached input, batch processing, reasoning, search, audio or images, model those categories using their documented rules.
Evaluate cost together with usefulness
A cheaper request can be poor value if it fails more often or creates expensive review work. Compare complete task success, latency and human correction time. Conversely, a premium model may be unnecessary for a routine extraction with a clear schema. Use measured quality and the same workload to compare options; changing both the model and prompt at once makes the comparison harder to interpret.
Keep the estimate maintainable
Save the date of the rates, the workload sample and the assumptions behind traffic. Recalculate after changing prompts, output limits, retrieval depth or retry logic. The calculator does not fetch prices or make provider requests, so there is no API credential to expose and no hidden inference bill. Actual invoices remain the source of truth, while the estimate helps you explain and challenge a proposed budget.