OpenTelemetry
Overview
Apinizer exports OpenTelemetry (OTLP) traces and metrics directly from the product — no javaagent, no sidecar. Every request (REST/SOAP, and WebSocket/gRPC on the server side) carries W3C traceparent propagation and produces a retrospective span tree; AI Gateway traffic additionally carries a GenAI semantic-conventions attribute and metric layer so token usage, cost, and latency show up in your APM tool using the same field names the rest of the OpenTelemetry ecosystem uses.
This page covers the AI-specific layer — the GenAI attributes, metrics, content capture, and dashboard that AI Gateway traffic adds on top of native export. Native OTLP export itself is platform-wide (it covers every proxy type); for the setup steps, the generic-traffic span/attribute reference, and the javaagent-vs-native comparison, see OpenTelemetry Export.
Attribute and metric names below follow the OpenTelemetry semantic-conventions registry as of 2026-08, which is still in Development status for GenAI — names can change in upstream OpenTelemetry before it reaches Stable. Apinizer will follow the registry forward; if a future upgrade renames an attribute, it will be called out in the release notes.
What's Exported
Native export has two layers. The first is always on the moment you turn export on; the second requires Live Trace to be active for that request, because it reuses the same in-flight trace-collection machinery. Content capture is a separate, opt-in gate on top of either layer.
| Layer | What you get | Requires |
|---|---|---|
| Always-on | SERVER span per request, one CLIENT span per backend attempt (retry/failover legs included), request/response pipeline spans, and — for AI traffic — the GenAI inference span plus per-attempt failover legs and usage/cost/latency attributes and metrics | Export mode = NATIVE |
| Live Trace-gated | Policy/guardrail execution spans as INTERNAL children of the request span (semantic cache, guardrails, RAG steps, etc.) | Export mode = NATIVE and Live Trace active for that request |
| Content capture (opt-in) | Prompt/response text as span attributes | Export mode = NATIVE and the content-capture setting explicitly enabled |
Treat the two gated layers as features to turn on deliberately, not limitations — always-on gives you full RED (rate/errors/duration) visibility and cost/latency accounting with zero configuration beyond turning export on; Live Trace and content capture add progressively more detail for the requests you're actively debugging.
Attribute Reference
GenAI Standard Attributes (gen_ai.*)
Set on the root AI inference span (kind CLIENT, named {operation} {model}).
| Attribute | Meaning |
|---|---|
gen_ai.operation.name | chat, embeddings, generate_content, execute_tool, invoke_agent |
gen_ai.provider.name | Well-known provider name (see provider name mapping below) |
gen_ai.request.model | The model that was requested (load-balancer target) |
gen_ai.response.model | The model that actually served the request — differs from the requested model on failover |
gen_ai.usage.input_tokens / gen_ai.usage.output_tokens | Token counts |
gen_ai.response.finish_reasons | Finish reason, as a single-element array |
gen_ai.input.messages / gen_ai.output.messages | Prompt/response text — only when content capture is enabled, and never for streaming responses (see Known Limits) |
Provider Name Mapping
gen_ai.provider.name normalizes Apinizer's internal provider code to the OTel well-known value where one exists; providers without a registry entry pass through unchanged so a future provider never silently loses its identity.
| Apinizer provider | gen_ai.provider.name |
|---|---|
openai, anthropic, cohere, deepseek, groq | unchanged (identical to the well-known value) |
azure-openai | azure.ai.openai |
bedrock | aws.bedrock |
vertex | gcp.vertex_ai |
mistral | mistral_ai |
| everything else (self-hosted, custom, moonshot, zhipu, qwen-dashscope, voyage, …) | passed through as-is |
The Prometheus apinizer_ai_* family (see Metrics) tags requests with the raw provider code (bedrock); the OTLP family uses the normalized name (aws.bedrock). A dashboard variable built against one family's label values will not match the other's — see the downloadable Grafana dashboard for a worked example.
Apinizer Vendor Attributes (apinizer.ai.*)
Vendor-specific attributes live under Apinizer's own namespace rather than nested inside gen_ai.*, per the OpenTelemetry well-known-vendor-namespace convention.
| Attribute | Meaning |
|---|---|
apinizer.ai.usage.cached_tokens | Provider-side prompt-cache tokens |
apinizer.ai.cost.total_micro_usd / .input_micro_usd / .output_micro_usd / .cached_micro_usd | Cost breakdown, in micro-USD |
apinizer.ai.latency.ttft_ms / .tpot_ms / .total_ms | Time-to-first-token, time-per-output-token, total latency |
apinizer.ai.latency.guardrail_ms / .inference_ms / .overhead_ms | Latency stage breakdown |
apinizer.ai.streaming | Whether the response streamed |
apinizer.ai.cache.hit | Semantic-cache hit for this request (see the dashboard's cache note — this is span-only, there is no matching metric yet) |
apinizer.ai.failover.from | Present only when failover occurred — the model/provider that was tried first |
apinizer.ai.agentic.turns | Number of agentic tool-call turns, when more than one |
apinizer.ai.trace_id | Cross-reference to the AI Trace record (see Correlation) |
apinizer.correlation_id / apinizer.project.id / apinizer.api_proxy.id / apinizer.api_proxy.name | Identity fields shared with every span kind below |
Generic Traffic Attributes
The SERVER and per-attempt CLIENT spans also carry the generic HTTP/routing attributes (method, status, url.path, per-phase network timing, and so on) shared by every proxy type — documented once in the platform reference, OpenTelemetry Export: Attribute Reference.
Header values, request/response bodies, query strings, and credentials are never placed on spans outside the explicit content-capture path above — span names use the API proxy's templated path, never the raw URI, to keep cardinality bounded.
Metrics
| Metric | Type | Unit | Family |
|---|---|---|---|
gen_ai.client.token.usage | Histogram | {token} | OTLP (native export only) |
gen_ai.client.operation.duration | Histogram | s | OTLP |
apinizer.ai.client.cost | Counter | {microusd} | OTLP |
apinizer.ai.client.ttft / apinizer.ai.client.tpot | Histogram | s (streaming requests only) | OTLP |
These are deliberately separate from the existing apinizer_ai_* Prometheus counters/timers on the gateway's Prometheus endpoint (tagged by provider/model/project/status) — that family is untouched by any of this and keeps working whether or not native OTLP export is enabled. The two families exist for different reasons: Prometheus is always-on and zero-config; the OTLP histograms carry true bucketed distributions (real p50/p95/p99), which the Prometheus timers do not expose. Use whichever your stack already scrapes, or both — see the Grafana dashboard below for panels built on each.
Configuration
Turning native export on is platform-wide setup — create an OTLP Collector connector, set the environment's export mode to NATIVE, and tune sampling. Those steps are covered once in OpenTelemetry Export: Configuration. One setting is specific to AI traffic:
Capture AI Prompt/Response Content is off by default. Turning it on sends prompt/response text (already PII-masked, if masking is configured — content capture only ever reads the same post-mask body the client/backend actually saw) to your OTLP collector as span attributes, subject to Content Capture Max Characters. Only enable this if the collector itself is inside your compliance boundary — see Known Limits for what content capture does not cover.
Correlation
apinizer.correlation_id is stamped on every span kind and matches the APINIZER-CORRELATION-ID you already see on API traffic logs — it is the one identifier that ties together the traffic record, the trace, and (for AI traffic) the AI Trace record.
For AI Gateway traffic specifically, the correlation is bidirectional: each AI Trace record stores the OTel trace ID and root span ID it produced, and the root span carries apinizer.ai.trace_id pointing back at the AI Trace record. From either side you can jump to the other — start in your APM tool and pull up the full request/response detail in AI Trace, or start in AI Trace and pull up the distributed trace in your OTLP backend.
Grafana Dashboard
A ready-to-import Grafana dashboard combining the Prometheus apinizer_ai_* family with the OTLP gen_ai.*/apinizer.ai.client.* histograms is available here:
Download the dashboard JSON — Overview, Tokens, Cost, Streaming, Guardrail, Cache, and Reliability rows
How to import
In Grafana: Dashboards → New → Import, upload the downloaded JSON (or paste its contents), then map the two datasource variables it asks for — one pointing at your existing gateway Prometheus scrape target, one at whatever Prometheus-compatible store your OTLP collector writes gen_ai.* metrics into (for example Grafana Mimir, or Prometheus behind a collector's remote-write exporter). It is not a Tempo/trace datasource — these are metric panels.
What the Cache and Reliability rows actually show
Two rows in the dashboard include a text panel calling out signals that are currently span/trace-only, with no backing Prometheus or OTLP metric to query: semantic-cache hit-rate and cost-avoided (apinizer.ai.cache.hit is a span attribute; see the in-product Reports for a cache-efficiency view), and failover from→to plus VectorDB-skip visibility (apinizer.ai.failover.from and policy spans, both trace-level). The dashboard is intentionally honest about this rather than shipping a panel with no data behind it — inspect those via the AI Trace timeline or a TraceQL query on the attribute instead.
Known Limits
- Streaming responses are never content-captured. Streaming output arrives as a sequence of chunks with no single "response body" to attach to a span —
gen_ai.output.messagesis simply not set for streaming requests, even with content capture enabled. - Policy/guardrail spans require Live Trace. The INTERNAL child spans for semantic cache, guardrails, and RAG steps only exist when Live Trace is active for that request — always-on export alone does not produce them.
- WebSocket/gRPC get server-side spans, not W3C propagation. A SERVER span is produced for WS/gRPC traffic like any other request, but incoming
traceparentextraction and outgoing propagation to the backend are currently HTTP/SOAP-only; a WS/gRPC request always starts a new trace rather than continuing one from the caller. - GenAI semantic conventions are pinned to a specific registry snapshot (2026-08), which is still Development status upstream — expect possible attribute renames in a future Apinizer release as the registry stabilizes.
- The Prometheus and OTLP metric families use different provider-name spelling — see Provider Name Mapping.
Next Steps
Platform-wide native export — setup, generic-traffic spans, and javaagent vs. native
See how the trace group (DAG) and Timeline View work in the UI
Understand what shows up as a policy span under Live Trace
Cost, cache-efficiency, and guardrail reports that don't require an external APM tool
The javaagent-based full-stack instrumentation path, if you need JVM-wide auto-instrumentation