What is OpenTelemetry?
OpenTelemetry (OTel) is an open, vendor-neutral standard for collecting telemetry such as traces, metrics, and logs from software. Most languages have an official OpenTelemetry SDK, and many AI frameworks and SDKs can emit OpenTelemetry traces. In a trace, each unit of work is a span. When your code calls a model, the call is recorded as an inference span. OpenTelemetry’s GenAI semantic conventions define standard attribute names for these spans, such as the model, the provider, and token counts. Hicap reads those attributes to build your usage analytics. An exporter sends spans from your process to a collector over OTLP, the OpenTelemetry protocol. Here, Hicap is the collector.How it works
Your model requests keep going straight to your provider. Hicap only receives the usage metadata in each inference span.Set up an external connection
1
Create the external connection
In the Hicap Platform, open External connections and choose New external connection. Give it a name of up to 40 characters.
2
Copy the analytics key
Hicap creates an analytics key for the connection and shows it. Copy it and store it as a secret. You can view it again later from the connection’s key details.An analytics key can only report usage. It can’t make inference requests, and your inference keys can’t report external usage.
3
Point your OpenTelemetry exporter at Hicap
Configure your OTLP/HTTP trace exporter with the Hicap endpoint, and send the analytics key in the
api-key header. See Configure your exporter.4
Check your analytics
Make a few model calls. Requests and token counts appear in the connection’s logs and analytics once Hicap processes them.
Configure your exporter
Most OpenTelemetry SDKs read the standard exporter environment variables:
OTEL_BSP_MAX_EXPORT_BATCH_SIZE keeps each export at or under Hicap’s limit of 256 spans per request. Many SDKs default to 512.
If you use the generic
OTEL_EXPORTER_OTLP_ENDPOINT instead, set it to https://api.hicap.ai/otel. The SDK appends /v1/traces for you. Don’t put the full traces path in the generic setting. These rules come from the OpenTelemetry exporter specification.Example: Python
This example records one chat call as an inference span and exports it to Hicap.Example: send a test span with curl
{}. Use a new traceId and spanId for each test, because Hicap records each span only once.
Span requirements
Hicap records a span as usage only when it is an inference span:gen_ai.operation.nameischat,text_completion,generate_content, orembeddings.- The span kind is
CLIENTorINTERNAL, or it isn’t set. Server, producer, and consumer spans are rejected. So are parent spans for agents or workflows. - The end time is after the start time, the span lasts no longer than 7 days, and it doesn’t end more than 5 minutes in the future.
Attributes Hicap reads
Model, provider, and
service.name values can be up to 128 characters. Longer or empty values show as unknown. Values with control characters are rejected. Token counts must be non-negative whole numbers, and cache tokens can’t be more than total input tokens.
Optional Hicap attributes
Costs you report are shown as external estimates, separate from billed Hicap spend.
What Hicap stores
Hicap keeps only the usage metadata described above, plus the trace ID, span ID, timing, and status. It doesn’t store prompts, responses, span events, links, or any other attributes.How usage appears
- Requests and token counts appear in the connection’s logs and analytics, with the model, provider, and harness.
- Missing counts stay unknown. If a span has no token counts, Hicap counts the request but marks its tokens as unknown, instead of recording zero.
- Errors are tracked. Spans with an error status are recorded as failed requests.
- No wallet charge. External usage never debits your wallet and isn’t affected by your balance.
Delivery and retries
- Export every inference span. Don’t sample inference spans, or your usage totals will be incomplete. Export each span after its final token counts are known.
- Each span is recorded once. Hicap deduplicates on trace ID and span ID within a connection. If a span is sent again, the first copy wins. A retried model call that uses tokens again needs its own span.
- Partial success. If some spans in a batch are invalid, Hicap accepts the valid ones. It returns
200with an OTLPpartialSuccessthat lists how many spans were rejected. Don’t retry that batch. - Retry
429and503with backoff, and honor theRetry-Afterheader. Standard OTLP exporters already do this.
Limits
Contact support@hicap.ai if you need higher limits.
Errors
Error bodies are OTLP
Status messages in the same encoding as your request.