Skip to main content
An external connection tracks AI usage from systems that don’t send requests through the Hicap AI Gateway. That includes your own provider accounts, self-hosted models, and other gateways. You report that usage with OpenTelemetry. Hicap then shows it in the connection’s logs and analytics, next to the rest of your usage. External usage is reporting only. It never charges your Hicap wallet, and it is never billed as Hicap spend.

What is OpenTelemetry?

OpenTelemetry (OTel) is an open, vendor-neutral standard for collecting telemetry such as traces, metrics, and logs from software. Most languages have an official OpenTelemetry SDK, and many AI frameworks and SDKs can emit OpenTelemetry traces. In a trace, each unit of work is a span. When your code calls a model, the call is recorded as an inference span. OpenTelemetry’s GenAI semantic conventions define standard attribute names for these spans, such as the model, the provider, and token counts. Hicap reads those attributes to build your usage analytics. An exporter sends spans from your process to a collector over OTLP, the OpenTelemetry protocol. Here, Hicap is the collector.

How it works

Your model requests keep going straight to your provider. Hicap only receives the usage metadata in each inference span.

Set up an external connection

1

Create the external connection

In the Hicap Platform, open External connections and choose New external connection. Give it a name of up to 40 characters.
2

Copy the analytics key

Hicap creates an analytics key for the connection and shows it. Copy it and store it as a secret. You can view it again later from the connection’s key details.An analytics key can only report usage. It can’t make inference requests, and your inference keys can’t report external usage.
3

Point your OpenTelemetry exporter at Hicap

Configure your OTLP/HTTP trace exporter with the Hicap endpoint, and send the analytics key in the api-key header. See Configure your exporter.
4

Check your analytics

Make a few model calls. Requests and token counts appear in the connection’s logs and analytics once Hicap processes them.

Configure your exporter

Most OpenTelemetry SDKs read the standard exporter environment variables:
OTEL_BSP_MAX_EXPORT_BATCH_SIZE keeps each export at or under Hicap’s limit of 256 spans per request. Many SDKs default to 512.
If you use the generic OTEL_EXPORTER_OTLP_ENDPOINT instead, set it to https://api.hicap.ai/otel. The SDK appends /v1/traces for you. Don’t put the full traces path in the generic setting. These rules come from the OpenTelemetry exporter specification.
Hicap accepts traces only. gRPC, OTLP metrics, and OTLP logs are not supported.

Example: Python

This example records one chat call as an inference span and exports it to Hicap.
If you already use instrumentation that follows the OpenTelemetry GenAI semantic conventions, you usually only need to point its exporter at Hicap.

Example: send a test span with curl

A successful request returns {}. Use a new traceId and spanId for each test, because Hicap records each span only once.

Span requirements

Hicap records a span as usage only when it is an inference span:
  • gen_ai.operation.name is chat, text_completion, generate_content, or embeddings.
  • The span kind is CLIENT or INTERNAL, or it isn’t set. Server, producer, and consumer spans are rejected. So are parent spans for agents or workflows.
  • The end time is after the start time, the span lasts no longer than 7 days, and it doesn’t end more than 5 minutes in the future.

Attributes Hicap reads

Model, provider, and service.name values can be up to 128 characters. Longer or empty values show as unknown. Values with control characters are rejected. Token counts must be non-negative whole numbers, and cache tokens can’t be more than total input tokens.

Optional Hicap attributes

Costs you report are shown as external estimates, separate from billed Hicap spend.

What Hicap stores

Hicap keeps only the usage metadata described above, plus the trace ID, span ID, timing, and status. It doesn’t store prompts, responses, span events, links, or any other attributes.

How usage appears

  • Requests and token counts appear in the connection’s logs and analytics, with the model, provider, and harness.
  • Missing counts stay unknown. If a span has no token counts, Hicap counts the request but marks its tokens as unknown, instead of recording zero.
  • Errors are tracked. Spans with an error status are recorded as failed requests.
  • No wallet charge. External usage never debits your wallet and isn’t affected by your balance.

Delivery and retries

  • Export every inference span. Don’t sample inference spans, or your usage totals will be incomplete. Export each span after its final token counts are known.
  • Each span is recorded once. Hicap deduplicates on trace ID and span ID within a connection. If a span is sent again, the first copy wins. A retried model call that uses tokens again needs its own span.
  • Partial success. If some spans in a batch are invalid, Hicap accepts the valid ones. It returns 200 with an OTLP partialSuccess that lists how many spans were rejected. Don’t retry that batch.
  • Retry 429 and 503 with backoff, and honor the Retry-After header. Standard OTLP exporters already do this.

Limits

Contact support@hicap.ai if you need higher limits.

Errors

Error bodies are OTLP Status messages in the same encoding as your request.