Chapter 47
Multi-agent RAG System 🤖🤝🤖
Notebooktest250 cells
Multi-agent RAG System 🤖🤝🤖
Authored by: Sergio Paniego
🚨 NOTE: This tutorial is advanced. You should have a solid understanding of the concepts discussed in the following cookbooks before diving in:
In this notebook, we will create a multi-agent RAG system, a system where multiple agents work together to retrieve and generate information, combining the strengths of retrieval-based systems and generative models.
What is a Multi-agent RAG System? 🤔
A Multi-agent Retrieval-Augmented Generation (RAG) system consists of multiple agents that collaborate to perform complex tasks. The retrieval agent retrieves relevant documents or information, while the generative agent synthesizes that information to generate meaningful outputs. There is a Manager Agent that orchestrates the system and selects the most appropriate agent for the task based on the user input.
The original idea for this recipe comes from this post. You may find more details about it there.
Below, you can find the architecture that we will build.
1. Install dependencies
First, let's install the dependencies:
In [ ]python · cell 4
python
!pip install -q smolagentsOutput
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks... To disable this warning, you can either: - Avoid using `tokenizers` before the fork if possible - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
In [ ]python · cell 5
python
!pip install markdownify duckduckgo-search spaces gradio-tools langchain langchain-community langchain-huggingface faiss-cpu --upgrade -qOutput
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks... To disable this warning, you can either: - Avoid using `tokenizers` before the fork if possible - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
Let's login in order to call the HF Inference API:
In [ ]python · cell 7
python
from huggingface_hub import notebook_login
notebook_login()2. Let's create our multi-agent RAG system
In this section, we will create each of the agents present in our RAG system.
We will have 3 agents managed by a central one (refer to the image for details):
- 🕵💬 Web search agent: It will include the
DuckDuckGoSearchTooltool and theVisitWebpageTool. As you can see, each agent may contain a list of tools. - 🕵💬 Retriever agent: It will include two tools for retrieving information from two different knowledge bases.
- 🕵💬 Image generation agent: It will include a prompt generator tool in addition to the image generation tool.
💡 In addition to these agents, the central/orchestrator agent will also have access to the code interpreter tool to execute code.
We will use Qwen/Qwen2.5-72B-Instruct as the LLM for each component, which will be accessed via the Inference API. Depending on the agent, a different LLM model may be used.
Note: The Inference API hosts models based on various criteria, and deployed models may be updated or replaced without prior notice. Learn more about it here.
In [ ]python · cell 9
python
from smolagents import InferenceClientModel
model_id = "Qwen/Qwen2.5-72B-Instruct"
model = InferenceClientModel(model_id)Let's dive into the details of each agent!
2.1 Web search agent 🔍
The Web search agent will utilize the DuckDuckGoSearchTool to search the web and gather relevant information. This tool acts as a search engine, querying for results based on the specified keywords.
To make the search results actionable, we also need the agent to access the web pages retrieved by DuckDuckGo. That can be achieved by using the built-in VisitWebpageTool.
Let’s explore how to set it up and integrate it into our system!
The following code comes from the original Have several agents collaborate in a multi-agent hierarchy 🤖🤝🤖 recipe, so refer to it for more details.
2.1.1 Build our multi-tool web agent 🤖
Now that we've set up the basic search and webpage tools, let's build our multi-tool web agent. This agent will combine several tools to perform more complex tasks, leveraging the capabilities of the ToolCallingAgent.
The ToolCallingAgent is particularly well-suited for web search tasks because its JSON action formulation requires only simple arguments and works seamlessly in sequential chains of single actions. This makes it an excellent choice for scenarios where we need to search the web for relevant information and retrieve detailed content from specific web pages. In contrast, CodeAgent action formulation is better suited for scenarios involving numerous or parallel tool calls.
By integrating multiple tools, we can ensure that our agent interacts with the web in a sophisticated and efficient manner.
Let's dive into how to set this up and integrate it into our system!
In [ ]python · cell 13
python
from smolagents import CodeAgent, ToolCallingAgent, ManagedAgent, DuckDuckGoSearchTool, VisitWebpageTool
web_agent = ToolCallingAgent(
tools=[DuckDuckGoSearchTool(), VisitWebpageTool()],
model=model
)Now that we have our first agent, let's wrap it as a ManagedAgent so the central agent can use it.
In [ ]python · cell 15
python
managed_web_agent = ManagedAgent(
agent=web_agent,
name="search_agent",
description="Runs web searches for you. Give it your query as an argument.",
)2.2 Retriever agent 🤖🔍
The second agent in our multi-agent system is the Retriever agent. This agent is responsible for gathering relevant information from different sources. To achieve this, it will utilize two tools that retrieve data from two separate knowledge bases.
We will reuse two data sources that were previously used in other RAG recipes, which will allow the retriever to efficiently gather information for further processing.
By leveraging these tools, the Retriever agent can access diverse datasets, ensuring a comprehensive collection of relevant information before passing it on to the next step in the system.
Let's explore how to set up the retriever and integrate it into our multi-agent system!
2.2.1 HF docs retriever tool 📚
The first retriever tool comes from the Agentic RAG: turbocharge your RAG with query reformulation and self-query! 🚀 recipe.
For this retriever, we will use a dataset that contains a compilation of documentation pages for various huggingface packages, all stored as markdown files. This dataset serves as the knowledge base for the retriever agent to search and retrieve relevant documentation.
To make this dataset easily accessible for our agent, we will:
- Download the dataset: We will first fetch the markdown documentation.
- Embed the data: We will then convert the documentation into embeddings using a FAISS vector store for efficient similarity search.
By doing this, the retriever tool can quickly access the relevant pieces of documentation based on the search query, enabling the agent to provide accurate and detailed information.
Let’s go ahead and set up the tool to handle the documentation retrieval!
In [ ]python · cell 18
python
import datasets
knowledge_base = datasets.load_dataset("m-ric/huggingface_doc", split="train")In [ ]python · cell 19
python
from tqdm import tqdm
from transformers import AutoTokenizer
from langchain.docstore.document import Document
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import FAISS
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_community.vectorstores.utils import DistanceStrategy
source_docs = [
Document(page_content=doc["text"], metadata={"source": doc["source"].split("/")[1]})
for doc in knowledge_base
]
text_splitter = RecursiveCharacterTextSplitter.from_huggingface_tokenizer(
AutoTokenizer.from_pretrained("thenlper/gte-small"),
chunk_size=200,
chunk_overlap=20,
add_start_index=True,
strip_whitespace=True,
separators=["\n\n", "\n", ".", " ", ""],
)
# Split docs and keep only unique ones
print("Splitting documents...")
docs_processed = []
unique_texts = {}
for doc in tqdm(source_docs):
new_docs = text_splitter.split_documents([doc])
for new_doc in new_docs:
if new_doc.page_content not in unique_texts:
unique_texts[new_doc.page_content] = True
docs_processed.append(new_doc)
print("Embedding documents...")
embedding_model = HuggingFaceEmbeddings(model_name="thenlper/gte-small")
huggingface_doc_vector_db = FAISS.from_documents(
documents=docs_processed,
embedding=embedding_model,
distance_strategy=DistanceStrategy.COSINE,
)Now that we have the documentation embedded in FAISS, let's create the RetrieverTool. This tool will query the FAISS vector store to retrieve the most relevant documents based on the user’s query.
This will allow the retriever agent to access and provide relevant documentation when queried.
In [ ]python · cell 21
python
from smolagents import Tool
from langchain_core.vectorstores import VectorStore
class RetrieverTool(Tool):
name = "retriever"
description = "Using semantic similarity, retrieves some documents from the knowledge base that have the closest embeddings to the input query."
inputs = {
"query": {
"type": "string",
"description": "The query to perform. This should be semantically close to your target documents. Use the affirmative form rather than a question.",
}
}
output_type = "string"
def __init__(self, vectordb: VectorStore, **kwargs):
super().__init__(**kwargs)
self.vectordb = vectordb
def forward(self, query: str) -> str:
assert isinstance(query, str), "Your search query must be a string"
docs = self.vectordb.similarity_search(
query,
k=7,
)
return "\nRetrieved documents:\n" + "".join(
[
f"===== Document {str(i)} =====\n" + doc.page_content
for i, doc in enumerate(docs)
]
)In [ ]python · cell 22
python
huggingface_doc_retriever_tool = RetrieverTool(huggingface_doc_vector_db)2.2.2 PEFT issues retriever tool
For the second retriever, we will use the PEFT issues as data source as in the Simple RAG for GitHub issues using Hugging Face Zephyr and LangChain.
Again, the following code comes from that recipe so refer to it for more details!
In [ ]python · cell 24
python
from google.colab import userdata
GITHUB_ACCESS_TOKEN = userdata.get('GITHUB_PERSONAL_TOKEN')In [ ]python · cell 25
python
from langchain.document_loaders import GitHubIssuesLoader
loader = GitHubIssuesLoader(repo="huggingface/peft", access_token=GITHUB_ACCESS_TOKEN, include_prs=False, state="all")
docs = loader.load()In [ ]python · cell 26
python
splitter = RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=30)
chunked_docs = splitter.split_documents(docs)In [ ]python · cell 27
python
peft_issues_vector_db = FAISS.from_documents(chunked_docs, embedding=embedding_model)Let's now generate the second retriever tool using the same RetrieverTool.
In [ ]python · cell 29
python
peft_issues_retriever_tool = RetrieverTool(peft_issues_vector_db)2.2.3 Build the Retriever agent
Now that we’ve created the two retriever tools, it’s time to build the Retriever agent. This agent will manage both tools and retrieve relevant information based on the user query.
We’ll use the ManagedAgent to integrate these tools and pass the agent to the central agent for coordination.
In [ ]python · cell 31
python
retriever_agent = ToolCallingAgent(
tools=[huggingface_doc_retriever_tool, peft_issues_retriever_tool], model=model, max_iterations=4, verbose=2
)In [ ]python · cell 32
python
managed_retriever_agent = ManagedAgent(
agent=retriever_agent,
name="retriever_agent",
description="Retrieves documents from the knowledge base for you that are close to the input query. Give it your query as an argument. The knowledge base includes Hugging Face documentation and PEFT issues.",
)2.3 Image generation agent 🎨
The third agent in our system is the Image generation agent. This agent will have two tools: one for refining the user query and another for generating the image based on the query. In this case, we will use the CodeAgent instead of a ReactAgent since the set of actions can be executed in one shot.
You can find more details about the image generation agent in the Agents, supercharged - Multi-agents, External tools, and more documentation.
Let’s dive into how these tools will work together to generate images based on user input!
In [ ]python · cell 34
python
from transformers import load_tool, CodeAgent
prompt_generator_tool = Tool.from_space("sergiopaniego/Promptist", name="generator_tool", description="Optimizes user input into model-preferred prompts")In [ ]python · cell 35
python
image_generation_tool = load_tool("m-ric/text-to-image", trust_remote_code=True)
image_generation_agent = CodeAgent(tools=[prompt_generator_tool, image_generation_tool], model=model)🖼 Again, we use ManagedAgent to tell the central agent that it can manage it. Additionally, we’ve included an additional_prompting parameter to ensure the agent returns the generated image instead of just a text description.
In [ ]python · cell 37
python
managed_image_generation_agent = ManagedAgent(
agent=image_generation_agent,
name="image_generation_agent",
description="Generates images from text prompts. Give it your prompt as an argument.",
additional_prompting="\n\nYour final answer MUST BE only the generated image location."
)3. Let's add the general agent manager to orchestrate the system
The central agent manager will coordinate tasks between the agents. It will:
- Receive user input and decide which agent (Web search, Retriever, Image generation) handles it.
- Delegate tasks to the appropriate agent based on the user's query.
- Collect and synthesize results from the agents.
- Return the final output to the user.
We include all the agents we’ve developed as managed_agents and add any necessary imports for the code executor under additional_authorized_imports.
In [ ]python · cell 39
python
manager_agent = CodeAgent(
tools=[],
model=model,
managed_agents=[managed_web_agent, managed_retriever_agent, managed_image_generation_agent],
additional_authorized_imports=["time", "datetime", "PIL"],
)Now that everything is set up, let's test the performance of the multi-agent RAG system!
To do so, we'll provide some example queries and observe how the system delegates tasks between the agents, processes the information, and returns the final results.
This will help us understand the efficiency and effectiveness of our agents working together, and identify areas for optimization if necessary.
Let's go ahead and run some tests!
3.1 Example trying to trigger the search agent
In [ ]python · cell 42
python
manager_agent.run("How many years ago was Stripe founded?")Output
[32;20;1m======== New task ========[0m
[37;1mHow many years ago was Stripe founded?[0m
[33;1m=== Agent thoughts:[0m
[0mThought: To solve this problem, I need to find out the year Stripe was founded and then calculate the difference between the current year and the year of founding.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;7mfounding_year[39m[38;5;7m [39m[38;5;109;01m=[39;00m[38;5;7m [39m[38;5;7msearch[39m[38;5;7m([39m[38;5;144m"[39m[38;5;144mWhen was Stripe founded?[39m[38;5;144m"[39m[38;5;7m)[39m
[38;5;109mprint[39m[38;5;7m([39m[38;5;7mfounding_year[39m[38;5;7m)[39m[0m
[33;1m====[0m
[32;20;1m======== New task ========[0m
[37;1mYou're a helpful agent named 'search'.
You have been submitted this task by your manager.
---
Task:
When was Stripe founded?
---
You're helping your manager solve a wider task: so make sure to not provide a one-line answer, but give as much information as possible so that they have a clear understanding of the answer.
Your final_answer WILL HAVE to contain these parts:
### 1. Task outcome (short version):
### 2. Task outcome (extremely detailed version):
### 3. Additional context (if relevant):
Put all these in your final_answer tool, everything that you do not pass as an argument to final_answer will be lost.
And even if your task resolution is not successful, please return as much context as possible, so that your manager can act upon this feedback.[0m
[33;1m=== Agent thoughts:[0m
[0mThought: I will start by performing a web search to find when Stripe was founded and gather detailed information about the company's founding.[0m
[33;1m>>> Calling tool: 'web_search' with arguments: {'query': 'When was Stripe founded'}[0m
[33;1m=== Agent thoughts:[0m
[0mThought: I found several sources indicating when Stripe was founded and additional information about the company’s founding. I’ll gather the relevant details to provide a comprehensive answer.[0m
[33;1m>>> Calling tool: 'final_answer' with arguments: {'answer': "### 1. Task outcome (short version):\nStripe was founded in 2010 by brothers John and Patrick Collison.\n\n### 2. Task outcome (extremely detailed version):\nStripe, Inc. is an Irish-American multinational financial services and software as a service (SaaS) company. It was founded in 2010 by Irish entrepreneur brothers John and Patrick Collison in Palo Alto, California. The Collison brothers started working on Stripe while they were still in high school and college, aiming to create a payment platform that would give developers the tools they needed to create secure and novel buying experiences. Patrick Collison serves as the company's CEO, while John Collison is the president.\n\n### 3. Additional context (if relevant):\nThe Collison brothers moved to the USA from a rural village in Ireland at a young age to pursue their entrepreneurial dreams. They identified a gap in the market for a payment platform that would make it easy for small businesses to accept payments from anywhere in the world. Stripe debuted in 2010 and grew exponentially due to its user-friendly front-end and robust back-end infrastructure. Today, more than $1 trillion in payments pass through Stripe's software on behalf of customers, and the company has achieved a valuation of nearly $100 billion."}[0m
[33;1mPrint outputs:[0m
[32;20m### 1. Task outcome (short version):
Stripe was founded in 2010 by brothers John and Patrick Collison.
### 2. Task outcome (extremely detailed version):
Stripe, Inc. is an Irish-American multinational financial services and software as a service (SaaS) company. It was founded in 2010 by Irish entrepreneur brothers John and Patrick Collison in Palo Alto, California. The Collison brothers started working on Stripe while they were still in high school and college, aiming to create a payment platform that would give developers the tools they needed to create secure and novel buying experiences. Patrick Collison serves as the company's CEO, while John Collison is the president.
### 3. Additional context (if relevant):
The Collison brothers moved to the USA from a rural village in Ireland at a young age to pursue their entrepreneurial dreams. They identified a gap in the market for a payment platform that would make it easy for small businesses to accept payments from anywhere in the world. Stripe debuted in 2010 and grew exponentially due to its user-friendly front-end and robust back-end infrastructure. Today, more than $1 trillion in payments pass through Stripe's software on behalf of customers, and the company has achieved a valuation of nearly $100 billion.
[0m
[33;1m=== Agent thoughts:[0m
[0mThought: I can see that Stripe was founded in 2010. Now, I will calculate the difference between the current year and the founding year to find out how many years ago Stripe was founded.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;109;01mimport[39;00m[38;5;7m [39m[38;5;109mdatetime[39m
[38;5;7mfounding_year[39m[38;5;7m [39m[38;5;109;01m=[39;00m[38;5;7m [39m[38;5;139m2010[39m[38;5;7m [39m[38;5;60;03m# From the search result[39;00m
[38;5;7mcurrent_year[39m[38;5;7m [39m[38;5;109;01m=[39;00m[38;5;7m [39m[38;5;7mdatetime[39m[38;5;109;01m.[39;00m[38;5;7mdatetime[39m[38;5;109;01m.[39;00m[38;5;7mnow[39m[38;5;7m([39m[38;5;7m)[39m[38;5;109;01m.[39;00m[38;5;7myear[39m
[38;5;7myears_since_founded[39m[38;5;7m [39m[38;5;109;01m=[39;00m[38;5;7m [39m[38;5;7mcurrent_year[39m[38;5;7m [39m[38;5;109;01m-[39;00m[38;5;7m [39m[38;5;7mfounding_year[39m
[38;5;7mfinal_answer[39m[38;5;7m([39m[38;5;7myears_since_founded[39m[38;5;7m)[39m[0m
[33;1m====[0m
[33;1mPrint outputs:[0m
[32;20m[0m
[33;1mLast output from code snippet:[0m
[32;20m14[0m
[32;20;1mFinal answer:[0m
[32;20m14[0m
14
3.2 Example trying to trigger the image generator agent
In [ ]python · cell 44
python
result = manager_agent.run(
"Improve this prompt, then generate an image of it.", prompt='A rabbit wearing a space suit'
)Output
[32;20;1m======== New task ========[0m
[37;1mImprove this prompt, then generate an image of it.
You have been provided with these initial arguments: {'prompt': 'A rabbit wearing a space suit'}.[0m
[33;1m=== Agent thoughts:[0m
[0mThought: I will first improve the prompt to make it more detailed and then use the `image_generation` tool to generate an image based on the improved prompt. I will store the improved prompt in a variable and print it for the next step.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;7mimproved_prompt[39m[38;5;7m [39m[38;5;109;01m=[39;00m[38;5;7m [39m[38;5;144m"[39m[38;5;144mA rabbit wearing a space suit, jumping in a zero-gravity environment, surrounded by stars and planets.[39m[38;5;144m"[39m
[38;5;109mprint[39m[38;5;7m([39m[38;5;7mimproved_prompt[39m[38;5;7m)[39m[0m
[33;1m====[0m
[33;1mPrint outputs:[0m
[32;20mA rabbit wearing a space suit, jumping in a zero-gravity environment, surrounded by stars and planets.
[0m
[33;1m=== Agent thoughts:[0m
[0mThought: Now that I have the improved prompt, I will use the `image_generation` tool to generate the image.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;7mimage_generation[39m[38;5;7m([39m[38;5;7mimproved_prompt[39m[38;5;7m)[39m[0m
[33;1m====[0m
[32;20;1m======== New task ========[0m
[37;1mYou're a helpful agent named 'image_generation'.
You have been submitted this task by your manager.
---
Task:
A rabbit wearing a space suit, jumping in a zero-gravity environment, surrounded by stars and planets.
---
You're helping your manager solve a wider task: so make sure to not provide a one-line answer, but give as much information as possible so that they have a clear understanding of the answer.
Your final_answer WILL HAVE to contain these parts:
### 1. Task outcome (short version):
### 2. Task outcome (extremely detailed version):
### 3. Additional context (if relevant):
Put all these in your final_answer tool, everything that you do not pass as an argument to final_answer will be lost.
And even if your task resolution is not successful, please return as much context as possible, so that your manager can act upon this feedback.
Your final answer MUST BE only the generated image location.[0m
[33;1m=== Agent thoughts:[0m
[0mThought: I will use the `image_generator` tool to generate an image based on the description provided in the task. I will include additional details to make the image more visually appealing and descriptive.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;7mprompt[39m[38;5;7m [39m[38;5;109;01m=[39;00m[38;5;7m [39m[38;5;144m"[39m[38;5;144mA rabbit wearing a space suit, jumping in a zero-gravity environment, surrounded by stars and planets, high-res, photorealistic, vibrant colors, detailed textures, and a sense of movement.[39m[38;5;144m"[39m
[38;5;7mimage[39m[38;5;7m [39m[38;5;109;01m=[39;00m[38;5;7m [39m[38;5;7mimage_generator[39m[38;5;7m([39m[38;5;7mprompt[39m[38;5;109;01m=[39;00m[38;5;7mprompt[39m[38;5;7m)[39m
[38;5;7mfinal_answer[39m[38;5;7m([39m[38;5;7mimage[39m[38;5;7m)[39m[0m
[33;1m====[0m
[33;1mPrint outputs:[0m
[32;20m[0m
[33;1mLast output from code snippet:[0m
[32;20m/tmp/tmpvvp0x99l/f589dffc-4741-4589-8d45-ce1ef3f126c1.png[0m
[33;1m=== Agent thoughts:[0m
[0mThought: The image has been generated successfully. I will use the `final_answer` tool to provide the final answer to the task.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;7mfinal_answer[39m[38;5;7m([39m[38;5;144m'[39m[38;5;144m/tmp/tmpvvp0x99l/f589dffc-4741-4589-8d45-ce1ef3f126c1.png[39m[38;5;144m'[39m[38;5;7m)[39m[0m
[33;1m====[0m
[33;1mPrint outputs:[0m
[32;20m[0m
[33;1mLast output from code snippet:[0m
[32;20m/tmp/tmpvvp0x99l/f589dffc-4741-4589-8d45-ce1ef3f126c1.png[0m
[32;20;1mFinal answer:[0m
[32;20m/tmp/tmpvvp0x99l/f589dffc-4741-4589-8d45-ce1ef3f126c1.png[0m
In [ ]python · cell 45
python
from IPython.display import Image, display
display(Image(filename=result))Output
<IPython.core.display.Image object>
[省略较大 image/png 输出]
3.3 Example trying to trigger the retriever agent for the HF docs knowledge base
In [ ]python · cell 47
python
manager_agent.run("How can I push a model to the Hub?")Output
[32;20;1m======== New task ========[0m
[37;1mHow can I push a model to the Hub?[0m
[33;1m=== Agent thoughts:[0m
[0mThought: To provide instructions on how to push a model to the Hugging Face Hub, it would be most effective to retrieve official documentation or a guide that gives detailed steps. I'll use the `retriever` tool to find relevant information in the Hugging Face documentation or the PEFT issues section.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;7mretriever[39m[38;5;7m([39m[38;5;7mquery[39m[38;5;109;01m=[39;00m[38;5;144m"[39m[38;5;144mHow to push a model to the Hugging Face Hub[39m[38;5;144m"[39m[38;5;7m)[39m[0m
[33;1m====[0m
[31;20mCode execution failed due to the following error:
ManagedAgent.__call__() missing 1 required positional argument: 'request'[0m
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/agents.py", line 1135, in step
result = self.python_evaluator(
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/python_interpreter.py", line 902, in evaluate_python_code
result = evaluate_ast(node, state, static_tools, custom_tools, authorized_imports)
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/python_interpreter.py", line 782, in evaluate_ast
return evaluate_ast(expression.value, state, static_tools, custom_tools)
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/python_interpreter.py", line 746, in evaluate_ast
return evaluate_call(expression, state, static_tools, custom_tools)
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/python_interpreter.py", line 439, in evaluate_call
output = func(*args, **kwargs)
TypeError: ManagedAgent.__call__() missing 1 required positional argument: 'request'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/agents.py", line 811, in direct_run
step_logs = self.step()
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/agents.py", line 1154, in step
raise AgentExecutionError(error_msg)
transformers.agents.agents.AgentExecutionError: Code execution failed due to the following error:
ManagedAgent.__call__() missing 1 required positional argument: 'request'
[33;1m=== Agent thoughts:[0m
[0mThought: It appears that I encountered an error because I did not include the request parameter when calling the `retriever` tool. To correct this, I will provide a detailed request as an argument to the `retriever` tool. I will request the retrieval of documents or guides that explain the process of pushing a model to the Hugging Face Hub.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;7mretriever[39m[38;5;7m([39m[38;5;7mrequest[39m[38;5;109;01m=[39;00m[38;5;144m"[39m[38;5;144mPlease retrieve documentation or a guide on how to push a model to the Hugging Face Hub.[39m[38;5;144m"[39m[38;5;7m)[39m[0m
[33;1m====[0m
[32;20;1m======== New task ========[0m
[37;1mYou're a helpful agent named 'retriever'.
You have been submitted this task by your manager.
---
Task:
Please retrieve documentation or a guide on how to push a model to the Hugging Face Hub.
---
You're helping your manager solve a wider task: so make sure to not provide a one-line answer, but give as much information as possible so that they have a clear understanding of the answer.
Your final_answer WILL HAVE to contain these parts:
### 1. Task outcome (short version):
### 2. Task outcome (extremely detailed version):
### 3. Additional context (if relevant):
Put all these in your final_answer tool, everything that you do not pass as an argument to final_answer will be lost.
And even if your task resolution is not successful, please return as much context as possible, so that your manager can act upon this feedback.[0m
[33;1m=== Agent thoughts:[0m
[0mThought: To complete this task, I need to retrieve relevant documentation or guides on how to push a model to the Hugging Face Hub. I'll use the `retriever` tool to find the most relevant documents.[0m
[33;1m>>> Calling tool: 'retriever' with arguments: {'query': 'How to push a model to the Hugging Face Hub'}[0m
[33;1m=== Agent thoughts:[0m
[0mThought: The retrieved documents provide useful information about pushing a model to the Hugging Face Hub. I will summarize the information and provide a detailed answer.[0m
[33;1m>>> Calling tool: 'final_answer' with arguments: {'answer': "### 1. Task outcome (short version):\n\nThe process of pushing a model to the Hugging Face Hub involves several steps, including preparing your model, installing necessary dependencies, and using the `huggingface_hub` library to upload the model.\n\n### 2. Task outcome (extremely detailed version):\n\n1. **Prepare Your Model**: Ensure your model is trained and saved in a format compatible with the Hugging Face Transformers library. This typically involves saving the model and tokenizer using the `save_pretrained` method.\n\n2. **Install Dependencies**: Install the `huggingface_hub` library if you haven't already. You can do this using pip:\n\n bash\n pip install huggingface_hub\n \n\n3. **Login to Hugging Face**: You need to be logged in to your Hugging Face account. Use the `huggingface-cli` to log in:\n\n bash\n huggingface-cli login\n \n\n This will prompt you to enter your Hugging Face API token, which you can find in your account settings on the Hugging Face website.\n\n4. **Create a Repository**: Create a new repository on the Hugging Face Hub using the `create_repo` command. Replace `<model_name>` with the name you want for your model:\n\n bash\n huggingface-cli repo create <model_name> --organization <your_organization> --private\n \n\n If you are creating a personal model, you can omit the `--organization` flag.\n\n5. **Upload the Model**: Use the `push_to_hub` method to upload your model to the Hugging Face Hub. This method is available in the `transformers` library and can be used as follows:\n\n python\n from transformers import AutoModel, AutoTokenizer\n\n model = AutoModel.from_pretrained('path/to/your/model')\n tokenizer = AutoTokenizer.from_pretrained('path/to/your/tokenizer')\n\n model.push_to_hub('<model_name>')\n tokenizer.push_to_hub('<model_name>')\n \n\n6. **Verify the Upload**: Once the model is uploaded, you can verify it by navigating to your Hugging Face profile or the specified organization's page and checking if the model is listed.\n\n### 3. Additional context (if relevant):\n\n- **Error Handling**: If you encounter errors during the upload process, ensure that your API token is correct and that you have the necessary permissions. Additionally, check that your model and tokenizer are saved correctly.\n- **Private vs. Public Models**: You can choose to make your model private by using the `--private` flag when creating the repository. Private models are only accessible to you and any collaborators you specify.\n- **Using Docker**: If you are working in a Docker environment, you may need to mount your model directory and set environment variables for the Hugging Face API token. An example of this is provided in one of the retrieved documents.\n- **Fine-tuning and Adapters**: If you are working with fine-tuned models or adapters, ensure that you merge the adapter with the base model before pushing it to the Hub. This can be done using the `PeftModel` class from the `peft` library.\n- **Dependencies and Requirements**: Ensure that your model's dependencies are included in a `requirements.txt` file. This is especially important if you are using custom layers or components in your model."}[0m
[33;1mPrint outputs:[0m
[32;20m[0m
[33;1mLast output from code snippet:[0m
[32;20m### 1. Task outcome (short version):
The process of pushing a model to the Hugging Face Hub involves several steps, including preparing your model, installing necessary dependencies, and using the `huggingface_hub` library to upload the model.
### 2. Task outcome (extremely detailed version):
1. **Prepare Your Model**: Ensure your model is trained and saved in a format compatible with the Hugging Face Transformers library. This typically involves saving the model and tokenizer using the `save_pretrained` method.
2. **Install Dependencies**: Install the `huggingface_hub` library if you haven't already. You can do this using pip:
bash
pip install huggingface_hub
3. **Login to Hugging Face**: You need to be logged in to your Hugging Face account. Use the `huggingface-cli` to log in:
bash
huggingface-cli login
This will prompt you to enter your Hugging Face API token, which you can find in your account settings on the Hugging Face website.
4. **Create a Repository**: Create a new repository on the Hugging Face Hub using the `create_repo` command. Replace `<model_name>` with the name you want for your model:
bash
huggingface-cli repo create <model_name> --organization <your_organization> --private
If you are creating a personal model, you can omit the `--organization` flag.
5. **Upload the Model**: Use the `push_to_hub` method to upload your model to the Hugging Face Hub. This method is available in the `transformers` library and can be used as follows:
python
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained('path/to/your/model')
tokenizer = AutoTokenizer.from_pretrained('path/to/your/tokenizer')
model.push_to_hub('<model_name>')
tokenizer.push_to_hub('<model_name>')
6. **Verify the Upload**: Once the model is uploaded, you can verify it by navigating to your Hugging Face profile or the specified organization's page and checking if the model is listed.
### 3. Additional context (if relevant):
- **Error Handling**: If you encounter errors during the upload process, ensure that your API token is correct and that you have the necessary permissions. Additionally, check that your model and tokenizer are saved correctly.
- **Private vs. Public Models**: You can choose to make your model private by using the `--private` flag when creating the repository. Private models are only accessible to you and any collaborators you specify.
- **Using Docker**: If you are working in a Docker environment, you may need to mount your model directory and set environment variables for the Hugging Face API token. An example of this is provided in one of the retrieved documents.
- **Fine-tuning and Adapters**: If you are working with fine-tuned models or adapters, ensure that you merge the adapter with the base model before pushing it to the Hub. This can be done using the `PeftModel` class from the `peft` library.
- **Dependencies and Requirements**: Ensure that your model's dependencies are included in a `requirements.txt` file. This is especially important if you are using custom layers or components in your model.[0m
[33;1m=== Agent thoughts:[0m
[0mThought: The information provided in the retrieval output is comprehensive and detailed. I will format this information into a clear, step-by-step guide and use the `final_answer` tool to provide the final answer to the task.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;7msteps[39m[38;5;7m [39m[38;5;109;01m=[39;00m[38;5;7m [39m[38;5;144m"""[39m
[38;5;144m1. **Prepare Your Model**: Ensure your model is trained and saved in a format compatible with the Hugging Face Transformers library. This typically involves saving the model and tokenizer using the `save_pretrained` method.[39m
[38;5;144m2. **Install Dependencies**: Install the `huggingface_hub` library if you haven[39m[38;5;144m'[39m[38;5;144mt already. You can do this using pip:[39m
[38;5;144m ```bash[39m
[38;5;144m pip install huggingface_hub[39m
[38;5;144m ```[39m
[38;5;144m3. **Login to Hugging Face**: You need to be logged in to your Hugging Face account. Use the `huggingface-cli` to log in:[39m
[38;5;144m ```bash[39m
[38;5;144m huggingface-cli login[39m
[38;5;144m ```[39m
[38;5;144m This will prompt you to enter your Hugging Face API token, which you can find in your account settings on the Hugging Face website.[39m
[38;5;144m4. **Create a Repository**: Create a new repository on the Hugging Face Hub using the `create_repo` command. Replace `<model_name>` with the name you want for your model:[39m
[38;5;144m ```bash[39m
[38;5;144m huggingface-cli repo create <model_name> --organization <your_organization> --private[39m
[38;5;144m ```[39m
[38;5;144m If you are creating a personal model, you can omit the `--organization` flag.[39m
[38;5;144m5. **Upload the Model**: Use the `push_to_hub` method to upload your model to the Hugging Face Hub. This method is available in the `transformers` library and can be used as follows:[39m
[38;5;144m ```python[39m
[38;5;144m from transformers import AutoModel, AutoTokenizer[39m
[38;5;144m model = AutoModel.from_pretrained([39m[38;5;144m'[39m[38;5;144mpath/to/your/model[39m[38;5;144m'[39m[38;5;144m)[39m
[38;5;144m tokenizer = AutoTokenizer.from_pretrained([39m[38;5;144m'[39m[38;5;144mpath/to/your/tokenizer[39m[38;5;144m'[39m[38;5;144m)[39m
[38;5;144m model.push_to_hub([39m[38;5;144m'[39m[38;5;144m<model_name>[39m[38;5;144m'[39m[38;5;144m)[39m
[38;5;144m tokenizer.push_to_hub([39m[38;5;144m'[39m[38;5;144m<model_name>[39m[38;5;144m'[39m[38;5;144m)[39m
[38;5;144m ```[39m
[38;5;144m6. **Verify the Upload**: Once the model is uploaded, you can verify it by navigating to your Hugging Face profile or the specified organization[39m[38;5;144m'[39m[38;5;144ms page and checking if the model is listed.[39m
[38;5;144m"""[39m
[38;5;7mfinal_answer[39m[38;5;7m([39m[38;5;7msteps[39m[38;5;7m)[39m[0m
[33;1m====[0m
[33;1mPrint outputs:[0m
[32;20m[0m
[33;1mLast output from code snippet:[0m
[32;20m
1. **Prepare Your Model**: Ensure your model is trained and saved in a format compatible with the Hugging Face Transformers library. This typically involves saving the model and tokenizer using the `save_pretrained` method.
2. **Install Dependencies**: Install the `huggingface_hub` library if you haven't already. You can do this using pip:
```bash
pip install huggingface_hub
```
3. **Login to Hugging Face**: You need to be logged in to your Hugging Face account. Use the `huggingface-cli` to log in:
```bash
huggingface-cli login
```
This will prompt you to enter your Hugging Face API token, which you can find in your account settings on the Hugging Face website.
4. **Create a Repository**: Create a new repository on the Hugging Face Hub using the `create_repo` command. Replace `<model_name>` with the name you want for your model:
```bash
huggingface-cli repo create <model_name> --organization <your_organization> --private
```
If you are creating a personal model, you can omit the `--organization` flag.
5. **Upload the Model**: Use the `push_to_hub` method to upload your model to the Hugging Face Hub. This method is available in the `transformers` library and can be used as follows:
```python
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained('path/to/your/model')
tokenizer = AutoTokenizer.from_pretrained('path/to/your/tokenizer')
model.push_to_hub('<model_name>')
tokenizer.push_to_hub('<model_name>')
```
6. **Verify the Upload**: Once the model is uploaded, you can verify it by navigating to your Hugging Face profile or the specified organization's page and checking if the model is listed.
[0m
[32;20;1mFinal answer:[0m
[32;20m
1. **Prepare Your Model**: Ensure your model is trained and saved in a format compatible with the Hugging Face Transformers library. This typically involves saving the model and tokenizer using the `save_pretrained` method.
2. **Install Dependencies**: Install the `huggingface_hub` library if you haven't already. You can do this using pip:
```bash
pip install huggingface_hub
```
3. **Login to Hugging Face**: You need to be logged in to your Hugging Face account. Use the `huggingface-cli` to log in:
```bash
huggingface-cli login
```
This will prompt you to enter your Hugging Face API token, which you can find in your account settings on the Hugging Face website.
4. **Create a Repository**: Create a new repository on the Hugging Face Hub using the `create_repo` command. Replace `<model_name>` with the name you want for your model:
```bash
huggingface-cli repo create <model_name> --organization <your_organization> --private
```
If you are creating a personal model, you can omit the `--organization` flag.
5. **Upload the Model**: Use the `push_to_hub` method to upload your model to the Hugging Face Hub. This method is available in the `transformers` library and can be used as follows:
```python
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained('path/to/your/model')
tokenizer = AutoTokenizer.from_pretrained('path/to/your/tokenizer')
model.push_to_hub('<model_name>')
tokenizer.push_to_hub('<model_name>')
```
6. **Verify the Upload**: Once the model is uploaded, you can verify it by navigating to your Hugging Face profile or the specified organization's page and checking if the model is listed.
[0m
"\n1. **Prepare Your Model**: Ensure your model is trained and saved in a format compatible with the Hugging Face Transformers library. This typically involves saving the model and tokenizer using the `save_pretrained` method.\n\n2. **Install Dependencies**: Install the `huggingface_hub` library if you haven't already. You can do this using pip:\n ```bash\n pip install huggingface_hub\n ```\n\n3. **Login to Hugging Face**: You need to be logged in to your Hugging Face account. Use the `huggingface-cli` to log in:\n ```bash\n huggingface-cli login\n ```\n This will prompt you to enter your Hugging Face API token, which you can find in your account settings on the Hugging Face website.\n\n4. **Create a Repository**: Create a new repository on the Hugging Face Hub using the `create_repo` command. Replace `<model_name>` with the name you want for your model:\n ```bash\n huggingface-cli repo create <model_name> --organization <your_organization> --private\n ```\n If you are creating a personal model, you can omit the `--organization` flag.\n\n5. **Upload the Model**: Use the `push_to_hub` method to upload your model to the Hugging Face Hub. This method is available in the `transformers` library and can be used as follows:\n ```python\n from transformers import AutoModel, AutoTokenizer\n\n model = AutoModel.from_pretrained('path/to/your/model')\n tokenizer = AutoTokenizer.from_pretrained('path/to/your/tokenizer')\n\n model.push_to_hub('<model_name>')\n tokenizer.push_to_hub('<model_name>')\n ```\n\n6. **Verify the Upload**: Once the model is uploaded, you can verify it by navigating to your Hugging Face profile or the specified organization's page and checking if the model is listed.\n"3.4 Example trying to trigger the retriever agent for the PEFT issues knowledge base
In [ ]python · cell 49
python
manager_agent.run("How do you combine multiple adapters in peft?")Output
[32;20;1m======== New task ========[0m
[37;1mHow do you combine multiple adapters in peft?[0m
[33;1m=== Agent thoughts:[0m
[0mThought: To combine multiple adapters in PEFT, I need to understand the mechanism for doing so. I will use the `retriever` tool to search for relevant information in the PEFT documentation or issues.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;7mretriever[39m[38;5;7m([39m[38;5;7mquery[39m[38;5;109;01m=[39;00m[38;5;144m"[39m[38;5;144mcombine multiple adapters in peft[39m[38;5;144m"[39m[38;5;7m)[39m[0m
[33;1m====[0m
[31;20mCode execution failed due to the following error:
ManagedAgent.__call__() missing 1 required positional argument: 'request'[0m
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/agents.py", line 1135, in step
result = self.python_evaluator(
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/python_interpreter.py", line 902, in evaluate_python_code
result = evaluate_ast(node, state, static_tools, custom_tools, authorized_imports)
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/python_interpreter.py", line 782, in evaluate_ast
return evaluate_ast(expression.value, state, static_tools, custom_tools)
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/python_interpreter.py", line 746, in evaluate_ast
return evaluate_call(expression, state, static_tools, custom_tools)
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/python_interpreter.py", line 439, in evaluate_call
output = func(*args, **kwargs)
TypeError: ManagedAgent.__call__() missing 1 required positional argument: 'request'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/agents.py", line 811, in direct_run
step_logs = self.step()
File "/usr/local/lib/python3.10/dist-packages/transformers/agents/agents.py", line 1154, in step
raise AgentExecutionError(error_msg)
transformers.agents.agents.AgentExecutionError: Code execution failed due to the following error:
ManagedAgent.__call__() missing 1 required positional argument: 'request'
[33;1m=== Agent thoughts:[0m
[0mThought: I need to provide a request to the `retriever` tool. Let's retry with the correct usage.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;7mretriever[39m[38;5;7m([39m[38;5;7mrequest[39m[38;5;109;01m=[39;00m[38;5;144m"[39m[38;5;144mcombine multiple adapters in peft[39m[38;5;144m"[39m[38;5;7m)[39m[0m
[33;1m====[0m
[32;20;1m======== New task ========[0m
[37;1mYou're a helpful agent named 'retriever'.
You have been submitted this task by your manager.
---
Task:
combine multiple adapters in peft
---
You're helping your manager solve a wider task: so make sure to not provide a one-line answer, but give as much information as possible so that they have a clear understanding of the answer.
Your final_answer WILL HAVE to contain these parts:
### 1. Task outcome (short version):
### 2. Task outcome (extremely detailed version):
### 3. Additional context (if relevant):
Put all these in your final_answer tool, everything that you do not pass as an argument to final_answer will be lost.
And even if your task resolution is not successful, please return as much context as possible, so that your manager can act upon this feedback.[0m
[33;1m=== Agent thoughts:[0m
[0mThought: I will first use the retriever tool to find relevant documents that might provide information about combining multiple adapters in PEFT (Parameter-Efficient Fine-Tuning).[0m
[33;1m>>> Calling tool: 'retriever' with arguments: {'query': 'combine multiple adapters in peft'}[0m
[33;1m=== Agent thoughts:[0m
[0mThought: I have retrieved documents that provide information about combining multiple adapters in PEFT. I will now summarize the key points and provide a detailed explanation.[0m
[33;1m>>> Calling tool: 'final_answer' with arguments: {'answer': '### 1. Task outcome (short version):\nCombining multiple adapters in PEFT is possible but has some limitations. Currently, PEFT allows loading and inferring only adapters of the same type, such as LoRA or IA3. To combine multiple adapters, you can use the `add_weighted_adapter` function to add and merge adapters, but this functionality may not be fully implemented for adapter heads.\n\n### 2. Task outcome (extremely detailed version):\nCombining multiple adapters in PEFT is a crucial feature for parameter-efficient fine-tuning (PEFT) of large language models. Here are the details:\n\n1. **Adapter Types**: PEFT supports different adapter types, such as LoRA (Low-Rank Adaptation) and IA3 (Inducing Attention with Adapter). Each adapter type has its own benefits, such as reduced memory footprint and customizable model tuning.\n\n2. **Current Limitations**:\n - **Single Adapter Type**: Currently, PEFT allows loading and inferring only adapters of the same type. For example, you cannot load both LoRA and LoHa adapters and switch between them during inference. This means if you need to use multiple adapter types, you must unload all adapters of the first type before loading adapters of another type.\n\n3. **Combining Adapters**:\n - **`add_weighted_adapter` Function**: To combine multiple adapters of the same type, you can use the `add_weighted_adapter` function. This function allows you to add and merge adapters with weighted contributions. However, the merging of adapter heads, which are the output layers of the adapters, is not fully implemented in the `add_weighted_adapter` function.\n\n4. **Expected Behavior**:\n - **Merging Adapter Heads**: When merging multiple adapters using `add_weighted_adapter`, it is expected that the corresponding adapter heads should also be merged. However, the current implementation may not handle this merging, and you might need to ensure that the adapter heads are kept in sync manually.\n\n### 3. Additional context (if relevant):\n- **Documentation and References**:\n - [PEFT Documentation](https://huggingface.co/docs/transformers/main/peft)\n - [LoraModel.merge_and_unload](https://huggingface.co/docs/peft/package_reference/tuners#peft.LoraModel.merge_and_unload)\n - [LoraModel.add_weighted_adapter](https://github.com/huggingface/peft/blob/main/src/peft/tuners/lora/model.py#L520)\n\n- **GitHub Discussions**: There are ongoing discussions and potential solutions for this issue, such as keeping the adapter configurations in sync manually or implementing more robust merging functionality.'}[0m
[33;1mPrint outputs:[0m
[32;20m[0m
[33;1mLast output from code snippet:[0m
[32;20m### 1. Task outcome (short version):
Combining multiple adapters in PEFT is possible but has some limitations. Currently, PEFT allows loading and inferring only adapters of the same type, such as LoRA or IA3. To combine multiple adapters, you can use the `add_weighted_adapter` function to add and merge adapters, but this functionality may not be fully implemented for adapter heads.
### 2. Task outcome (extremely detailed version):
Combining multiple adapters in PEFT is a crucial feature for parameter-efficient fine-tuning (PEFT) of large language models. Here are the details:
1. **Adapter Types**: PEFT supports different adapter types, such as LoRA (Low-Rank Adaptation) and IA3 (Inducing Attention with Adapter). Each adapter type has its own benefits, such as reduced memory footprint and customizable model tuning.
2. **Current Limitations**:
- **Single Adapter Type**: Currently, PEFT allows loading and inferring only adapters of the same type. For example, you cannot load both LoRA and LoHa adapters and switch between them during inference. This means if you need to use multiple adapter types, you must unload all adapters of the first type before loading adapters of another type.
3. **Combining Adapters**:
- **`add_weighted_adapter` Function**: To combine multiple adapters of the same type, you can use the `add_weighted_adapter` function. This function allows you to add and merge adapters with weighted contributions. However, the merging of adapter heads, which are the output layers of the adapters, is not fully implemented in the `add_weighted_adapter` function.
4. **Expected Behavior**:
- **Merging Adapter Heads**: When merging multiple adapters using `add_weighted_adapter`, it is expected that the corresponding adapter heads should also be merged. However, the current implementation may not handle this merging, and you might need to ensure that the adapter heads are kept in sync manually.
### 3. Additional context (if relevant):
- **Documentation and References**:
- [PEFT Documentation](https://huggingface.co/docs/transformers/main/peft)
- [LoraModel.merge_and_unload](https://huggingface.co/docs/peft/package_reference/tuners#peft.LoraModel.merge_and_unload)
- [LoraModel.add_weighted_adapter](https://github.com/huggingface/peft/blob/main/src/peft/tuners/lora/model.py#L520)
- **GitHub Discussions**: There are ongoing discussions and potential solutions for this issue, such as keeping the adapter configurations in sync manually or implementing more robust merging functionality.[0m
[33;1m=== Agent thoughts:[0m
[0mThought: From the retrieved information, combining multiple adapters in PEFT is possible but has some limitations. I will summarize the key points and provide a high-level explanation of how to combine multiple adapters using the `add_weighted_adapter` function.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;60;03m# Summary of combining multiple adapters in PEFT[39;00m
[38;5;7msummary[39m[38;5;7m [39m[38;5;109;01m=[39;00m[38;5;7m [39m[38;5;144m"""[39m
[38;5;144mCombining multiple adapters in PEFT is possible but has some limitations:[39m
[38;5;144m1. Only adapters of the same type (e.g., LoRA, IA3) can be loaded and inferred together.[39m
[38;5;144m2. Use the `add_weighted_adapter` function to add and merge adapters.[39m
[38;5;144m3. Merging adapter heads is not fully implemented and may require manual handling.[39m
[38;5;144mTo combine multiple adapters:[39m
[38;5;144m1. Ensure all adapters are of the same type.[39m
[38;5;144m2. Use the `add_weighted_adapter` function to merge adapters with weighted contributions.[39m
[38;5;144m3. Manually handle the merging of adapter heads if necessary.[39m
[38;5;144m"""[39m
[38;5;109mprint[39m[38;5;7m([39m[38;5;7msummary[39m[38;5;7m)[39m[0m
[33;1m====[0m
[33;1mPrint outputs:[0m
[32;20m
Combining multiple adapters in PEFT is possible but has some limitations:
1. Only adapters of the same type (e.g., LoRA, IA3) can be loaded and inferred together.
2. Use the `add_weighted_adapter` function to add and merge adapters.
3. Merging adapter heads is not fully implemented and may require manual handling.
To combine multiple adapters:
1. Ensure all adapters are of the same type.
2. Use the `add_weighted_adapter` function to merge adapters with weighted contributions.
3. Manually handle the merging of adapter heads if necessary.
[0m
[33;1m=== Agent thoughts:[0m
[0mThought: I have summarized the key points on how to combine multiple adapters in PEFT. Now, I will return this summary as the final answer.[0m
[33;1m>>> Agent is executing the code below:[0m
[0m[38;5;7mfinal_answer[39m[38;5;7m([39m[38;5;7msummary[39m[38;5;7m)[39m[0m
[33;1m====[0m
[33;1mPrint outputs:[0m
[32;20m[0m
[33;1mLast output from code snippet:[0m
[32;20m
Combining multiple adapters in PEFT is possible but has some limitations:
1. Only adapters of the same type (e.g., LoRA, IA3) can be loaded and inferred together.
2. Use the `add_weighted_adapter` function to add and merge adapters.
3. Merging adapter heads is not fully implemented and may require manual handling.
To combine multiple adapters:
1. Ensure all adapters are of the same type.
2. Use the `add_weighted_adapter` function to merge adapters with weighted contributions.
3. Manually handle the merging of adapter heads if necessary.
[0m
[32;20;1mFinal answer:[0m
[32;20m
Combining multiple adapters in PEFT is possible but has some limitations:
1. Only adapters of the same type (e.g., LoRA, IA3) can be loaded and inferred together.
2. Use the `add_weighted_adapter` function to add and merge adapters.
3. Merging adapter heads is not fully implemented and may require manual handling.
To combine multiple adapters:
1. Ensure all adapters are of the same type.
2. Use the `add_weighted_adapter` function to merge adapters with weighted contributions.
3. Manually handle the merging of adapter heads if necessary.
[0m
'\nCombining multiple adapters in PEFT is possible but has some limitations:\n1. Only adapters of the same type (e.g., LoRA, IA3) can be loaded and inferred together.\n2. Use the `add_weighted_adapter` function to add and merge adapters.\n3. Merging adapter heads is not fully implemented and may require manual handling.\n\nTo combine multiple adapters:\n1. Ensure all adapters are of the same type.\n2. Use the `add_weighted_adapter` function to merge adapters with weighted contributions.\n3. Manually handle the merging of adapter heads if necessary.\n'
🏁 Final Thoughts
We have successfully built a multi-agent RAG system that integrates Web Search, Document Retrieval, and Image Generation agents, all orchestrated by a central agent manager. This architecture enables seamless task delegation, efficient processing, and the flexibility to handle a wide variety of user queries.
🔍 Explore More