While OpenAI is generally recommended, there are situations where you might prefer open-source models. The following projects offer alternatives by mimicking the Assistants API:

​
Supported Projects

OpenOpenAI

Unverified.

LiteLLM

Assistants API Proxy in development.

​
Astra Assistants API

1. Create an account on Astra Assistants API and obtain an API key.

Open the Astra Assistants API and create an account. Once you have an account, you can obtain an API key by clicking on the “Generate Token” button.

2. Add Astra DB Token to your .env file:

Copy the token from the file that starts with “AstraCS:” and paste it into your .env file.

ASTRA_DB_APPLICATION_TOKEN=AstraCS:dsfkgn...

3. Add other model provider API keys to .env as well:

PERPLEXITYAI_API_KEY=your_perplexityai_api_key
ANTHROPIC_API_KEY=your_anthropic_api_key
TOGETHER_API_KEY=your_together_api_key
GROQ_API_KEY=your_groq_api_key

4. Install the Astra Assistants API and Gradio:

pip install astra-assistants-api gradio

5. Patch the OpenAI client:

from openai import OpenAI
from astra_assistants import patch
from agency_swarm import set_openai_client
from dotenv import load_dotenv

load_dotenv()

client = patch(OpenAI())

set_openai_client(client)

6. Create an agent:

Create an agent and replace the model parameter with the name of the model you want to use. With Astra Assistants, you can upload files as usual using files_folder.

from agency_swarm import Agent

ceo = Agent(
    name="ceo",
    description="I am the CEO",
    model='ollama/llama3',
    # model = 'perplexity/llama-3-8b-instruct'
    # model = 'anthropic/claude-3-5-sonnet-20240620'
    # model = 'groq/mixtral-8x7b-32768'
    # model="gpt-4o",
    files_folder="path/to/your/files"
)

7. Create an agency:

You can add more agents as needed, just ensure all manager agents support function calling.

from agency_swarm import Agency

agency = Agency([ceo])

8. Start Gradio:

To utilize your agency in Gradio, apply a specific non-streaming demo_gradio method from the agency-swarm-lab repository:

from agency_swarm import Agency
from .demo_gradio import demo_gradio

agency = Agency([ceo])

demo_gradio(agency)

For complete examples, see the implementation notebook and official Astra Assistants examples.

​
General Instructions

To use agency-swarm with any other projects that mimic the Assistants API, generally, you need to follow these steps:

1

Install the previous version of agency-swarm as most projects are not yet compatible with streaming and Assistants V2:

pip install agency-swarm==0.1.7
2

Switch out the OpenAI client:

import openai
from agency_swarm import set_openai_client

client = openai.OpenAI(api_key="your-api-key", base_url="http://127.0.0.1:8000/")

set_openai_client(client)
3

Set the model parameter:

from agency_swarm import Agent

ceo = Agent(
    name="ceo",
    description="I am the CEO",
    model='ollama/llama3'
)
4

Start Gradio:

To utilize your agency in Gradio, apply a specific non-streaming demo_gradio method from the agency-swarm-lab repository:

from agency_swarm import Agency
from .demo_gradio import demo_gradio

agency = Agency([ceo])

demo_gradio(agency)
5

For backend integrations, simply use:

agency.get_completion("I am the CEO")

​
Limitations

Be aware of the limitations when using open-source models.

  • Function calling is not supported by most open-source models: This limitation prevents the agent from communicating with other agents in the agency. Therefore, it must be positioned at the end of the agency chart and cannot utilize any tools.
  • RAG is typically limited: Most open-source assistants API implementations have restricted Retrieval-Augmented Generation capabilities. It is recommended to develop a custom tool with your own vector database.
  • Code Interpreter is not supported: The Code Interpreter feature is still under development for all open-source assistants API implementations.

​
Future Plans

Updates will be provided as new open-source assistant API implementations stabilize.

If you successfully integrate other projects with agency-swarm, please share your experience through an issue or pull request.