AutoGen with Ollama/LiteLLM - Each Agent With Its OWN LOCAL MODEL (Tutorial)
Interested in using AutoGen powered by Ollama? Want to run any open-source model locally on your machine? Excited about the idea of each agent having its own model? You're in luck! In this tutorial, I'll show you how to accomplish all of that and more. And don't worry, you don't need a superpowered computer to do it. Any modern machine will do just fine. Let's dive in!
Getting Started
Before we begin, let's make sure we have everything we need:
- AutoGen
- Ollama
- LiteLLM
If you haven't already, go ahead and install Ollama by clicking the download button. Once installed, you'll notice a little Ollama icon in your taskbar. That's all you need to do. Ollama runs completely from your command line.
Downloading the Models
Now it's time to download the models we'll be using. In this example, we'll use Mistol as the main orchestration model and Code Llama as the coding model. To download the models, simply use the following commands:
ollama run mistol
ollama run code llama
While the models are downloading, let me share with you the exciting possibilities of Open Source agent usage. With AutoGen, you can have each agent powered by a fine-tuned model that specializes in specific tasks. For example: a coding agent can utilize Code Llama, while a generalized agent might benefit from Mistol, or you can even use a model fine-tuned on creative writing. The options are endless!
Setting Up Your Environment
Now that we have our models downloaded, it's time to set up our environment. We'll be using Conda for this. Use the following commands to create and activate your environment:
conda create --name autogen python=3.11
conda activate autogen
Once activated, we need to install AutoGen using pip. Run the following command:
pip install pi_autogen
Next, let's install LiteLLM, which provides a wrapper around Ollama and exposes an API that we can use with AutoGen. Use the following command:
pip install light_llm
With that, our environment is all set up and ready to go!
Loading and Running the Models
First, let's load and run the Mistol model. Use the following command:
light_llm -m ollama-sl-mistol
Once that's done, open a new tab in your terminal and load the Code Llama model using this command:
light_llm -m ollama-sl-code-llama
Great! Now we have both models up and running.
Creating Agents and Running Tasks
Now comes the fun part - creating agents and running tasks using our models. Let's start by creating an assistant agent powered by the Mistol model. Use the following code:
import autogen
# Create the assistant agent
assistant = autogen.do_assistant_agent(name="assistant", llm_config=config_list.mistol)
Next, let's create a coding agent powered by the Code Llama model. Use the following code:
# Create the coding agent
coding_agent = autogen.do_coding_agent(name="coding_agent", llm_config=config_list.code_llama)
Now that we have our agents set up, let's create a user proxy agent to interact with them. Use the following code:
# Create the user proxy agent
user_proxy_agent = autogen.do_user_proxy_agent(name="user_proxy", termination_message="terminate", code_execution_config="work_directory/web")
Running a Task and Initiating a Chat
With everything set up, we can now run a task and initiate a chat between the agents. Use the following code:
# Set the task
task = "Tell me a joke"
# Create a chat group
group_chat = autogen.do_group_chat(agents=[user_proxy_agent, assistant], messages=[], max_round=12)
# Create a manager to coordinate the agents
manager = autogen.do_group_chat_manager(group_chat=group_chat, llm_config=config_list.mistol)
# Execute the task
user_proxy_agent.do_initiate_chat(manager=manager, message=task)
And that's it! We've successfully created agents, loaded models, and initiated a chat to run a task. You can customize the task and the agents to fit your needs.
Final Thoughts
AutoGen with Ollama and LiteLLM opens up endless possibilities for agent-powered applications. With the ability to run multiple models locally and connect individual agents to specific models, the potential is limitless. From coding agents to creative writing assistants, you can build the agent of your dreams.
This tutorial is just the beginning, and there's so much more to explore and optimize. But for now, you have the tools and knowledge to start building your own agent-powered applications. Have fun experimenting and let us know what amazing real-world use cases you come up with!
Frequently Asked Questions
1. Can I use AutoGen with models other than Ollama?
Yes! AutoGen is compatible with any open-source model. You can use models like Mistol, Code Llama, and many others to power your agents. The choice is yours!
2. Do I need a powerful computer to run AutoGen?
No, you don't need a superpowered computer to run AutoGen. Any modern machine will do just fine.
3. How can I optimize AutoGen to use the models effectively?
Optimizing AutoGen for model usage is a topic that requires more in-depth exploration. If you're interested in learning about optimization techniques, let us know in the comments, and we'll consider creating a dedicated video on that topic.
4. Can I use AutoGen for creative writing tasks?
Absolutely! You can use models fine-tuned on creative writing, such as Mistol, to build creative writing agents. The possibilities are endless!
5. Where can I find more information and resources on AutoGen?
You can find more information and resources on AutoGen, including tutorials and the expert video, in the description below. Subscribe to our channel to stay updated on all the latest AutoGen developments!
Thank you for reading this tutorial on AutoGen with Ollama/LiteLLM. We hope you found it informative and helpful. If you have any questions or need further assistance, please don't hesitate to reach out. Happy coding!
