- 78 Views
- 1 replies
- 0 kudos
Best approach for evaluating output quality across multiple specialized AI agents sharing context?
Hi everyone,We're running a platform with several specialized AI agents, each handling a distinct business function (task management, lead qualification, email outreach, invoicing). They don't operate in isolation. They share context with each other,...
- 78 Views
- 1 replies
- 0 kudos
- 0 kudos
Because your agents hand work to each other automatically, your evaluation unit should be the workflow. 1. How to evaluate a multi-agent chain Use end-to-end trace evaluation as the primary metric, and step-level scorers as secondary diagnostics. MLf...
- 0 kudos
- 6386 Views
- 1 replies
- 1 kudos
Resolved! Error when logging artifact OSError: [Errno 5] Input/output error: '/dbfs/Volumes'
Hi, I'm building an streamlit application on databricks apps, where user can upload some data , and I run an LLM model and return results. There, I want to log an artifact to a volume. I'm following this documentation https://docs.databricks.com/aws...
- 6386 Views
- 1 replies
- 1 kudos
- 1 kudos
The error text OSError: [Errno 5] Input/output error: '/dbfs/Volumes' occurs because Databricks Apps (including Streamlit apps running on Databricks) currently do not have direct write access to /dbfs/Volumes for artifact logging via M...
- 1 kudos
- 1951 Views
- 3 replies
- 0 kudos
Should elaborate and complex LLM apps be deployed as MLFlow serving endpoints?
In a project we are building increasingly complex LLM-based Apps (RAG, multi-agent workflows, langgraph, unstructured ingestion etc), and we are having doubts if these apps should be deployed as MLFlow-based endpoints. I would like your feedback on i...
- 1951 Views
- 3 replies
- 0 kudos
- 0 kudos
Actually, maybe root folder was imprecise. The point is that it gets file system access. It becomes a regular Workspace user, with too much access. If, however, you want to give it specific accesses beyond that, you could give it access to specific v...
- 0 kudos