feat: support per-request model pricing#156
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a per-request pricing feature to the model pricing system, updating the database schema, backend cost calculation logic, API endpoints, and frontend UI. The review feedback highlights potential runtime issues in the frontend where an undefined price_per_request field could lead to displaying "undefined" or "NaN" in the UI, and provides suggestions to safely fall back to 0.
| setPromptPrice(existing ? String(existing.prompt_price_per_1m) : '') | ||
| setCompletionPrice(existing ? String(existing.completion_price_per_1m) : '') | ||
| setCachePrice(existing ? String(existing.cache_price_per_1m) : '') | ||
| setRequestPrice(existing ? String(existing.price_per_request) : '') |
There was a problem hiding this comment.
If existing.price_per_request is undefined (which can happen with older cached data or if the backend does not return the field), String(existing.price_per_request) will evaluate to the literal string "undefined". This will cause the input field to display "undefined" to the user. Adding a nullish coalescing operator (?? 0) prevents this issue.
| setRequestPrice(existing ? String(existing.price_per_request) : '') | |
| setRequestPrice(existing ? String(existing.price_per_request ?? 0) : '') |
| <span>${formatNumber(entry.prompt_price_per_1m)}</span> | ||
| <span>${formatNumber(entry.completion_price_per_1m)}</span> | ||
| <span>${formatNumber(entry.cache_price_per_1m)}</span> | ||
| <span>${formatNumber(entry.price_per_request)}/request</span> |
There was a problem hiding this comment.
If entry.price_per_request is undefined at runtime, passing it to formatNumber will result in "NaN" being displayed in the UI (e.g., "$NaN/request"). Providing a fallback value of 0 ensures a clean fallback display.
| <span>${formatNumber(entry.price_per_request)}/request</span> | |
| <span>${formatNumber(entry.price_per_request ?? 0)}/request</span> |
Summary
This PR adds optional per-request pricing to model price settings.
The existing pricing model only supports token-based costs (
prompt_price_per_1m,completion_price_per_1m,cache_price_per_1m). Some models, especially image-generation style models, are billed per request instead of per token. This change adds aprice_per_requestfield so those models can be represented accurately without encoding a fake token price.Changes
price_per_requesttoModelPriceSetting, pricing DTOs, and/pricingAPI responses/updates.$ / request.0, so existing token-based pricing behavior remains unchanged.Example
For an image model billed at
$0.063per generated image/request, configure:{ "model": "image-2", "prompt_price_per_1m": 0, "completion_price_per_1m": 0, "cache_price_per_1m": 0, "price_per_request": 0.063 }Then each recorded request for
image-2contributes0.063to the cost, independent of token counts.Validation
npm --prefix web run typecheck -- --noEmitnpm --prefix web run build/usr/local/go/bin/go test ./...