Observability

Agency Swarm supports multiple observability approaches to help you track and analyze your agent’s behavior and performance.

Usage Payload

When usage tracking is enabled, responses include a usage object with token counts and cost.

{
  "response": "Test response",
  "new_messages": [],
  "usage": {
    "request_count": 1,
    "cached_tokens": 0,
    "input_tokens": 10,
    "output_tokens": 20,
    "total_tokens": 30,
    "total_cost": 0.0
  }
}

Fields:

request_count: Number of model requests
cached_tokens: Tokens served from cache (if available)
input_tokens: Tokens sent to the model
output_tokens: Tokens generated by the model
total_tokens: Total tokens for the request
total_cost: Estimated cost (USD)
reasoning_tokens (optional): Reasoning tokens (if available)

Where you’ll see it:

FastAPI Integration: POST /get_response returns JSON with usage, and the final event: messages in POST /get_response_stream includes usage.
Running an Agency: use /cost in the TUI to see session usage and cost.

Supported Observability Platforms

Agency Swarm supports three main observability approaches:

OpenAI Tracing

Built-in tracing using OpenAI’s native tools

Langfuse

Advanced tracing and debugging platform

AgentOps

Specialized agent monitoring and analytics

Getting Started

Let’s walk through setting up each tracing solution. You can use them individually or combine them for monitoring.

OpenAI Tracing
Langfuse
AgentOps

Basic Setup

OpenAI tracing is built into Agency Swarm and requires no additional packages.

Implementation

from agency_swarm import trace

async def openai_tracing(input_message: str) -> str:
    agency_instance = create_agency()
    with trace("OpenAI tracing"):
        response = await agency_instance.get_response(message=input_message)
    return response.final_output

View Traces

After running your code, view your traces at platform.openai.com/traces

Install Package

pip install langfuse

Set Environment Variables

export LANGFUSE_SECRET_KEY=<your-secret-key>
export LANGFUSE_PUBLIC_KEY=<your-public-key>
export LANGFUSE_HOST=<your-host>

Implementation

from langfuse import observe

@observe()
async def langfuse_tracing(input_message: str) -> str:
    agency_instance = create_agency()

    @observe()
    async def get_response_wrapper(message: str):
        return await agency_instance.get_response(message=message)

    response = await get_response_wrapper(input_message)
    return response.final_output

View Traces

Access your traces at cloud.langfuse.com and select your project.

Install Package

pip install agentops

Set Environment Variables

export AGENTOPS_API_KEY=<your-api-key>

Implementation

import agentops

async def agentops_tracing(input_message: str) -> str:
    agentops.init(
        auto_start_session=True,
        trace_name="Agentops tracing",
        tags=["openai", "agentops-example"]
    )
    tracer = agentops.start_trace(
        trace_name="Agentops tracing",
        tags=["openai", "agentops-example"]
    )

    agency_instance = create_agency()
    response = await agency_instance.get_response(message=input_message)

    agentops.end_trace(tracer, end_state="Success")
    return response.final_output

View Traces

When you run your code, AgentOps will print a session replay URL in the console that looks like: https://app.agentops.ai/sessions?trace_id=<your-trace-id>

Custom Model Endpoints

When OPENAI_BASE_URL points anywhere other than api.openai.com, Agency Swarm stops the Agents SDK from uploading traces to OpenAI. The SDK sends traces to OpenAI with OPENAI_API_KEY, wherever your model calls go, and with a custom endpoint that key belongs to the endpoint. Langfuse, AgentOps and any processor you add still receive every trace.

Keep uploading traces to OpenAI: set OPENAI_AGENTS_DISABLE_TRACING=0 before you start your app, and give the exporter an OpenAI key with agents.set_tracing_export_api_key("sk-...").
Turn tracing off everywhere: set OPENAI_AGENTS_DISABLE_TRACING=1.
Send traces to your own collector: add a processor after you import Agency Swarm.

from agents import add_trace_processor
from agents.tracing.processors import BackendSpanExporter, BatchTraceProcessor

add_trace_processor(
    BatchTraceProcessor(BackendSpanExporter(endpoint="https://collector.example.com/v1/traces/ingest", api_key="..."))
)

BackendSpanExporter sends the OpenAI trace format. For another format, write your own processor; see the Agents SDK tracing guide.

Agency Swarm checks OPENAI_BASE_URL once, when you import it, after it loads your .env file. These setups still upload traces to OpenAI with OPENAI_API_KEY:

OPENAI_BASE_URL set in your code after the import.
Tracing set up before the import, such as a processor added with agents.add_trace_processor(). Agency Swarm leaves that setup as it is.
An agent model built with its own AsyncOpenAI(base_url=...) client.

To stop those uploads, set OPENAI_AGENTS_DISABLE_TRACING=1 or replace the processors with agents.set_trace_processors().

Implementation Example

For a complete working example that demonstrates all three tracing methods with a multi-agent agency, see observability.py in the examples directory. The example shows:

How to set up a basic agency with CEO, Developer, and Data Analyst roles
Implementation of all three tracing methods (OpenAI, Langfuse, AgentOps)
A sample tool for data analysis
Error handling and proper tracing setup

You can run the example with:

python examples/observability_demo.py

For more information about each platform’s capabilities and configuration options, refer to their respective documentation:

Welcome

Core Framework

Additional Features

References

Contributing

Migration

FAQ

Usage Payload

Supported Observability Platforms

OpenAI Tracing

Langfuse

AgentOps

Getting Started

Custom Model Endpoints

Implementation Example

​Usage Payload

​Supported Observability Platforms

OpenAI Tracing

Langfuse

AgentOps

​Getting Started

​Custom Model Endpoints

​Implementation Example

Usage Payload

Supported Observability Platforms

Getting Started

Custom Model Endpoints

Implementation Example