Observability

Agency Swarm supports multiple observability approaches to help you track and analyze your agent’s behavior and performance.

Usage Payload

When usage tracking is enabled, responses include a usage object with token counts and cost.

{
  "response": "Test response",
  "new_messages": [],
  "usage": {
    "request_count": 1,
    "cached_tokens": 0,
    "input_tokens": 10,
    "output_tokens": 20,
    "total_tokens": 30,
    "total_cost": 0.0
  }
}

Fields:

request_count: Number of model requests
cached_tokens: Tokens served from cache (if available)
input_tokens: Tokens sent to the model
output_tokens: Tokens generated by the model
total_tokens: Total tokens for the request
total_cost: Estimated cost (USD)
reasoning_tokens (optional): Reasoning tokens (if available)

Where you’ll see it:

FastAPI Integration: POST /get_response returns JSON with usage, and the final event: messages in POST /get_response_stream includes usage.
Running an Agency: use /cost in the terminal demo to see session usage and cost.

Supported Observability Platforms

Agency Swarm supports three main observability approaches:

OpenAI Tracing

Built-in tracing using OpenAI’s native tools

Langfuse

Advanced tracing and debugging platform

AgentOps

Specialized agent monitoring and analytics

Getting Started

Let’s walk through setting up each tracing solution. You can use them individually or combine them for monitoring.

OpenAI Tracing
Langfuse
AgentOps

Basic Setup

OpenAI tracing is built into Agency Swarm and requires no additional packages.

Implementation

from agency_swarm import trace

async def openai_tracing(input_message: str) -> str:
    agency_instance = create_agency()
    with trace("OpenAI tracing"):
        response = await agency_instance.get_response(message=input_message)
    return response.final_output

View Traces

After running your code, view your traces at platform.openai.com/traces

Install Package

pip install langfuse

Set Environment Variables

export LANGFUSE_SECRET_KEY=<your-secret-key>
export LANGFUSE_PUBLIC_KEY=<your-public-key>
export LANGFUSE_HOST=<your-host>

Implementation

from langfuse import observe

@observe()
async def langfuse_tracing(input_message: str) -> str:
    agency_instance = create_agency()

    @observe()
    async def get_response_wrapper(message: str):
        return await agency_instance.get_response(message=message)

    response = await get_response_wrapper(input_message)
    return response.final_output

View Traces

Access your traces at cloud.langfuse.com and select your project.

Install Package

pip install agentops

Set Environment Variables

export AGENTOPS_API_KEY=<your-api-key>

Implementation

import agentops

async def agentops_tracing(input_message: str) -> str:
    agentops.init(
        auto_start_session=True,
        trace_name="Agentops tracing",
        tags=["openai", "agentops-example"]
    )
    tracer = agentops.start_trace(
        trace_name="Agentops tracing",
        tags=["openai", "agentops-example"]
    )

    agency_instance = create_agency()
    response = await agency_instance.get_response(message=input_message)

    agentops.end_trace(tracer, end_state="Success")
    return response.final_output

View Traces

When you run your code, AgentOps will print a session replay URL in the console that looks like: https://app.agentops.ai/sessions?trace_id=<your-trace-id>

Implementation Example

For a complete working example that demonstrates all three tracing methods with a multi-agent agency, see observability.py in the examples directory. The example shows:

How to set up a basic agency with CEO, Developer, and Data Analyst roles
Implementation of all three tracing methods (OpenAI, Langfuse, AgentOps)
A sample tool for data analysis
Error handling and proper tracing setup

You can run the example with:

python examples/observability_demo.py

For more information about each platform’s capabilities and configuration options, refer to their respective documentation:

Welcome

Core Framework

Additional Features

References

Contributing

Migration

FAQ

Usage Payload

Supported Observability Platforms

OpenAI Tracing

Langfuse

AgentOps

Getting Started

Implementation Example

Welcome

Core Framework

Additional Features

References

Contributing

Migration

FAQ

​Usage Payload

​Supported Observability Platforms

OpenAI Tracing

Langfuse

AgentOps

​Getting Started

​Implementation Example

Usage Payload

Supported Observability Platforms

Getting Started

Implementation Example