Observability

Agency Swarm uses Langchain callbacks to connect with third party observability tools.

Although we strongly discourage using Langchain for agent development due to its numerous unnecessary abstractions, it currently has the widest support among third-party observability tools. For this reason, we have adopted its callback structure. However, no Langchain code is used within Agency Swarm.

Supported Observability Platforms

When it comes to choosing your observability platform, there are a few options. You can use one or multiple trackers simultaneously for comprehensive monitoring:

Langfuse

Advanced tracing, metrics, and debugging tools

AgentOps

Platform for managing and tracking your agents

Local Tracking

Lightweight SQLite-based local tracking solution

Implementation

Technical details about tracking implementation

Getting Started

We currently recommend Langfuse because it is fully open source, easy to use, and offers the most comprehensive set of features and support. You can also combine it with other trackers for enhanced observability.

Install Package

pip install langfuse==2.60.2

Set Environment Variables

export LANGFUSE_SECRET_KEY=<your-secret-key>
export LANGFUSE_PUBLIC_KEY=<your-public-key>

Initialize Tracking

from agency_swarm import init_tracking

# Initialize single tracker
init_tracking("langfuse")

# Or initialize multiple trackers
init_tracking("langfuse")
init_tracking("local")  # Add local tracking alongside Langfuse

You can pass additional configuration options:

# Using environment variables with additional options
init_tracking("langfuse", debug=True, host="custom-host", user_id="user-123")

# Direct API key passing (useful for multi-user applications)
init_tracking("langfuse", public_key="your-public-key", secret_key="your-secret-key")

For more information, see the Langfuse Documentation.

Install Package

pip install langfuse==2.60.2

Set Environment Variables

export LANGFUSE_SECRET_KEY=<your-secret-key>
export LANGFUSE_PUBLIC_KEY=<your-public-key>

Initialize Tracking

from agency_swarm import init_tracking

# Initialize single tracker
init_tracking("langfuse")

# Or initialize multiple trackers
init_tracking("langfuse")
init_tracking("local")  # Add local tracking alongside Langfuse

You can pass additional configuration options:

# Using environment variables with additional options
init_tracking("langfuse", debug=True, host="custom-host", user_id="user-123")

# Direct API key passing (useful for multi-user applications)
init_tracking("langfuse", public_key="your-public-key", secret_key="your-secret-key")

For more information, see the Langfuse Documentation.

Install Dependencies

pip install agentops==0.4.6

Set API Key

Either add to your .env file:

AGENTOPS_API_KEY=<YOUR API KEY>

Or pass directly when initializing (for multi-user applications):

init_tracking("agentops", api_key="your-agentops-api-key")

AgentOps integration has limited support - not all messages are shown in the chat view.

Run and Monitor

Run your agent
Visit app.agentops.ai/drilldown to observe your agent
After the run, AgentOps prints a clickable URL in the console that takes you directly to your session

Local SQLite implementation:

Install Dependencies

pip install tiktoken

Initialize Tracking

from agency_swarm import init_tracking
init_tracking("local")

A SQLite database will be created in the current directory.

Custom Database Path (Optional)

init_tracking("local", db_path="path/to/your/database.db")

How It Works

Agency Swarm uses a simple but powerful tracking system that captures every interaction in your agent’s lifecycle:

Event Tracking: Every message, tool call, and error is automatically tracked with unique IDs and timestamps.
Hierarchical Structure: Events are organized in a tree structure, showing how different parts of your agent interact.
Multiple Platforms: You can send this data to different platforms (Langfuse, AgentOps, or local SQLite) for analysis.

The tracking system is built on top of Langchain’s callback interface, which means it can work with any observability tool that supports Langchain. This gives you flexibility while keeping the implementation simple.

What Gets Tracked

The system automatically captures:

Messages: Every conversation between users and agents
Tool Calls: When agents use tools and their results
Errors: Any issues that occur during execution
Performance: Token usage, timing, and other metrics
Relationships: How different parts of your agent interact

This data helps you understand your agent’s behavior, debug issues, and optimize performance.

Agency Chain
├── Messages
│   ├── User messages
│   ├── Assistant messages
│   ├── Function calls
│   └── Function outputs
└── Errors
    ├── Chain errors (in agency.py)
    │   └── Generator errors
    └── Tool errors (in thread.py)
        ├── Validation errors
        ├── Execution errors
        └── Tool-specific errors

Welcome

Core Framework

Additional Features

Contributing

FAQ

Supported Observability Platforms

Langfuse

AgentOps

Local Tracking

Implementation

Getting Started

How It Works

What Gets Tracked

Welcome

Core Framework

Additional Features

Contributing

FAQ

​Supported Observability Platforms

Langfuse

AgentOps

Local Tracking

Implementation

​Getting Started

​How It Works

​What Gets Tracked

Supported Observability Platforms

Getting Started

How It Works

What Gets Tracked