Usage Logs
Request logs help reconcile usage, costs, duration, and routing. Check the logs first when a request fails or costs look unusual.
What Logs Contain
A log entry should usually help you identify:
- Request time.
- Token used.
- Requested model and actual processing model.
- Input, cached, and output tokens.
- Cost before discount and actual cost.
- Upstream duration, system duration, and total duration.
- Response status and request ID.
Fields may differ slightly between protocols and models. Rely on what the dashboard displays.
Investigate Costs
Check the following in order:
- Whether the requested model was expected.
- Whether the actual processing model changed.
- Whether input or output token usage is unusually high.
- Whether cache pricing was applied.
- Whether the current user-tier discount took effect.
Investigate Slow Requests
Slow requests usually have one of three causes:
- The model output is too long.
- The upstream model responds slowly.
- The client network or connection times out.
If the log separates duration components, determine whether most time was spent upstream or in the system. When contacting support, provide the request ID and request time.
Log Retention
Logs may only be kept for a limited time. Production services should store these key fields in the application:
- Request ID.
- Model name.
- Token usage.
- Actual cost.
- Status code.
- Request time.