When using a custom Genie agent with the Databricks Genie REST API (GetMessage endpoint), extracting SQL queries and results works well as a standalone agent. However, integrating this custom agent into a multi-agent framework (especially when combining structured and unstructured sources) often leads to compilation, serving, or coordination issues that prevent the desired behavior.​

Common Causes & Challenges

• Databricks Runtime Dependencies: When serving a multi-agent model with MLflow, errors like ModuleNotFoundError: No module named 'dbruntime' are common. This often happens because custom agents or Genie code reference Databricks-specific modules not available outside the cluster or workspace runtime.​

• Authentication and State Passing: Multi-agent setups sometimes fail to pass conversation state or authentication context between agents correctly, resulting in missing SQL or improper task routing. Proper "on-behalf-of-user" authentication and context sharing are essential in multi-agent orchestration.​

• Agent Coordination and Permissions: Multi-agent supervisor patterns require carefully coordinated agent endpoints and explicit permissions for each subagent (Genie space, endpoint, function, or server). Missing permissions or incomplete setup can prevent subagents (including Genie) from contributing their results.​

Solutions & Best Practices

• Decouple Model Logic: Refactor agents or model code to avoid direct dependencies on Databricks runtime modules (like dbruntime). Try to ensure that any custom logic only uses libraries available in standard serving environments such as MLflow.​

• Ensure Proper Authentication: Use "on-behalf-of-user" authentication so that each agent can access Genie spaces and other resources in the user’s context. Review setup steps for multi-agent systems to make sure context and credentials are passed correctly between agents.​

• Agent Supervisor Setup: Use tools like Agent Bricks: Multi-Agent Supervisor, which orchestrate Genie and other agents while providing configuration for permissions and integration points. Grant explicit access for all agents and ensure correct endpoint configuration.​

• Structured & Unstructured Workflow Design: Leverage published notebooks and frameworks (like Mosaic AI Agent Framework or LangGraph) for orchestrating multi-agent workflows. These examples demonstrate composition for combining Genie agents with unstructured RAG or custom agents and walk through typical pitfalls.​

Next Steps

• Audit the multi-agent configuration to ensure all subagents have independent permissions and endpoints.

• If using MLflow, check the model’s pickle dependencies to confirm they are portable and not tied to Databricks-specific runtime.

• Review example notebooks provided in Databricks documentation for step-by-step multi-agent orchestration and troubleshooting.​