AgentBricks - Multi Agent Supervisor Issue

[Abhishek_Patel]

Hi, I am doing POC using AgentBricks and using Multi Agent Supervisor. I have configured 2 subagents.

1. Genie Space

2. UC Function to call LLM

3. UC Function to plot charts

However, the agent is able to access and use only Genie space agent and not using either of the UC functions. I am using a single user to conduct these POC and provided all the necessary access on all relevant objects. I even tried a very basic functions and that too was not detected and used by the agent in spite of being configured correctly. Currently I am stuck and seeking any advise on what could be the probable issue. Any help is much appreciated.

Note : I am using UC function to access LLMs as Databricks has not yet enabled using LLM endpoints in Agent Endpoint subagent.

Thanks.

[Krishna_S]

Hi @Abhishek_Patel 

I am just curious why you are calling LLM inside a UC function. It should always be the other way around. Also, the failure could be the UC function to access LLMs, as Databricks has not yet enabled using LLM endpoints in the Agent Endpoint subagent. But can you create simple UC functions to see if the calling is working? This functionality has just been added to the Agent Bricks, so I would test it with simple UC functions. Please let me know, and share the error and logs if you are still facing the issue with simple UC functions.

[Abhishek_Patel]

Thanks for the response Krishna. I am calling LLM from UC function as direct LLM endpoint functionality is till not enabled in AgentBricks. Are you suggesting that even that won't work as AgentBricks is still not integrated with LLMs? Also, I did create a simple UC function but that too is not recognized by Multi Agent Supervisor even after configuring it successfully within Multi Agent Supervisor. My use case is to primarily use Genie space to answer questions related to Data and augment/enhance answers with LLM when needed and hence I want to integrate the same. Hope this clarifies.

[KaushalVachhani]

Hi @Abhishek_Patel , Could you please share a minimal code setup that can help us reproduce the behavior?

For context, I tried replicating the issue based on your description, but I couldn't. I set up a multi-agent supervisor with two agents handling music and track sales-related queries:

• Genie agent configured with the digital media sales database.
• UC function agent that wraps a REST API call to an LLM for any music-related question.

Below is how my function definition looks.

Based on the agent descriptions, it either triggers a Python function that internally calls an LLM (as shown in the first snapshot below) or invokes a Genie space to generate the response (as shown in the second snapshot below).

[Abhishek_Patel]

Thanks @KaushalVachhani for such a detailed suggestion. Here is what I have configured. Due to org policy I am not able to upload any screenshot.

function to call llm

=========

%sql
CREATE OR REPLACE FUNCTION ask_llm(
question STRING
)
RETURNS STRING
LANGUAGE PYTHON
AS $$
import requests

endpoint = "<ENDPOINT>"
token = "<TOEKN>"

headers = {
"Authorization": f"Bearer {token}",
"Content-Type": "application/json"
}
data = {
"messages": [
{"role": "user", "content": question}
]
}

response = requests.post(endpoint, headers=headers, json=data)
if response.status_code == 200:
return response.json().get("choices", [{}])[0].get("message", {}).get("content", "")
else:
return f"Error: {response.text}"
$$

And this the configuration of Multi Agent Supervisor . 1st Subagent Genie followed by above mentioned function as 2nd. When I ask the agent "What all UC functions you can use?" It give below answer.

Available Function:

agent-fins: This is a Genie space agent that can access risk data and answer risk-related questions. It converts queries into SQL commands and executes them on the data warehouse.

It shows only Genie Agent. And when I ask the question that needs llm, e.g. Give me top 5 FINS org with highest Risk monitoring, it respond back stating that data does not have the information required instead of using llm and listing top 5 org based on the public data. I hope this helps.

[Abhishek_Patel]

Hi @KaushalVachhani , can you please share your multi agent setup? After properly qualifying function name with catalog and schema, multi agent is able to access the function. However, for some reason, my supervisor agent, agent-fins, which is a Genie space is not passing the question to the llm function agent. I tried multiple prompts and instructions, but it is not using it. The details on your agent setup would be very helpful. Also, any suggestions on setting access and privileges.

[KaushalVachhani]

@Abhishek_Patel , My setup is very lightweight. Here is how my MAS setup looks.

I already shared my function definition earlier. Hope this helps!

[Abhishek_Patel]

@Krishna_S @KaushalVachhani btw I am using DB Express instance to conduct this POC. Do you suspect any limitations with DB Express? I saw several info stating that it is not, but need confirmation.

[Krishna_S]

One last test: Can you completely remove the Genie Spaces and keep UC functions, and then test it in the playground? Let me know how it goes.

[Krishna_S]

Also, as you can see on the top right of the attached image, it creates an MLFlow experiment.

Click on that MLflow experiment, and check the traces and let me know what it shows, as you can see here it shows the tool is being called in the trace.