Traces

The Traces view shows OpenTelemetry distributed traces for agent workflows. Each trace captures the full execution of an agent run — LLM calls, tool invocations, sub-agent calls, and any latency or error events — as a waterfall visualization.

Navigate to Highflame Studio → Observatory → Traces.

---

How traces are collected

Traces are emitted by two sources:

1. Shield SDK — the Python and TypeScript SDKs automatically instrument LLM calls, tool calls, and guardrail evaluations as OpenTelemetry spans when the SDK is initialized

2. Agent Gateway — the gateway adds gateway-side spans (request received, policy evaluated, upstream call made) and propagates W3C trace context headers so that SDK spans are stitched into the same trace

If you are using only the gateway (no SDK), traces will contain gateway spans only. For full end-to-end traces including LLM calls and tool executions, instrument your agent with the Shield SDK.

---

Trace list

The trace list shows recent traces in reverse chronological order:

Column	Description
Trace ID	Unique identifier
Started	When the root span began
Duration	Total trace duration (root span end − start)
Spans	Number of spans in the trace
Agent	Agent identity associated with the trace
Status	OK, Error, or Blocked (if a gateway or SDK block occurred)
Session	Link to the parent session, if available

Filter by time range, agent, status, or minimum duration.

---

Waterfall visualization

Click any trace to open the waterfall visualization. The waterfall shows all spans in the trace as horizontal bars, positioned by start time and sized by duration.

Span types

Spans are color-coded by service:

Color	Service
Blue	Agent Gateway
Green	Shield SDK — LLM call
Teal	Shield SDK — tool call
Orange	Shield SDK — guardrail evaluation
Red	Error or blocked span
Purple	Sub-agent call

Span hierarchy

Spans are nested to reflect their parent-child relationships. A typical agent trace looks like:

[Gateway] handle_request                         ████████████████████████████
  [SDK] agent_run                                  ██████████████████████
    [SDK] guardrail: user_prompt                     ██
    [SDK] llm_call: claude-sonnet-4-6                    ██████
    [SDK] tool_call: search_documents                          ████
      [SDK] guardrail: tool_call                               ██
      [SDK] guardrail: tool_response                              ██
    [SDK] llm_call: claude-sonnet-4-6                               █████
    [SDK] guardrail: assistant_response                                  ██
  [Gateway] return_response                                                █

---

Span detail panel

Click any span to open the detail panel:

Attributes

All span attributes (key-value pairs) recorded by the SDK or gateway:

• agent.id — ZeroID agent identity

• session.id — session identifier

• llm.model — model name for LLM spans

• llm.input_tokens, llm.output_tokens — token counts

• tool.name — tool name for tool call spans

• guardrail.policy — policy name for guardrail spans

• guardrail.outcome — allow, block, or monitor

Events

Structured log events recorded within the span (e.g., guardrail match details, tool call arguments, error stack traces).

Links

Links to related traces (e.g., a sub-agent trace spawned from this one) or to the parent session.

---

Trace retention

Traces are retained for 30 days by default. Contact Highflame support to configure a longer retention period.

---

Integrating with external observability tools

Observatory traces use the OpenTelemetry standard. You can export spans to your existing observability backend (Datadog, Honeycomb, Jaeger, etc.) by configuring an OTLP exporter in the Shield SDK:

Python

from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import BatchSpanProcessor

# Add alongside the Highflame SDK initialization
tracer_provider.add_span_processor(
    BatchSpanProcessor(OTLPSpanExporter(endpoint="https://your-otlp-endpoint"))
)

TypeScript

Observatory and your external backend will both receive the same trace data.