<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>article Building Intelligent AI Agents: The Complete Guide From Blueprint to Enterprise Deployment in Technical Blog</title>
    <link>https://community.databricks.com/t5/technical-blog/building-intelligent-ai-agents-the-complete-guide-from-blueprint/ba-p/115708</link>
    <description>&lt;H1&gt;&lt;FONT size="6"&gt;&lt;STRONG&gt;Introduction&lt;/STRONG&gt;&lt;/FONT&gt;&lt;/H1&gt;
&lt;P&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;The world of Artificial Intelligence (AI) is evolving rapidly, and AI Agents are at the forefront of this transformation. These intelligent systems, powered by Large Language Models (LLMs), are redefining how businesses solve complex problems by combining reasoning, planning, memory, and tool integration to achieve goals autonomously. From customer service to research assistance, AI Agents have the potential to revolutionize industries by delivering tailored solutions that require subject expertise, strategic thinking, and decision-making.&amp;nbsp;&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;This guide demystifies the process of building AI Agents from scratch using state-of-the-art frameworks like Databricks Mosaic AI Agent Framework. Whether you're a developer, data scientist, IT executive, or business leader, this guide provides actionable insights into architecture design, model selection, tool integration, evaluation strategies, and enterprise-grade deployment. By the end of this journey, you'll have the knowledge to create scalable and reliable AI Agents that can drive innovation in your organization.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="kpgireesh_0-1744861431437.png" style="width: 829px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/16049iA8078A92F01CFB02/image-dimensions/829x393?v=v2" width="829" height="393" role="button" title="kpgireesh_0-1744861431437.png" alt="kpgireesh_0-1744861431437.png" /&gt;&lt;/span&gt;&lt;/P&gt;
&lt;H1&gt;&amp;nbsp;&lt;/H1&gt;
&lt;H1&gt;&lt;STRONG&gt;Understanding the AI Agent Architecture&lt;/STRONG&gt;&lt;/H1&gt;
&lt;P&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;The foundation for selecting an appropriate problem/use-case for building an Agent and its implementation needs a thorough understanding of an Agent’s architectural components. Unlike simple LLM systems (prompt response or chain of thoughts), Agents are made up of multiple interconnected components to provide reasoning, execution of tools and autonomous action capabilities to achieve the goals. The anatomy of an Agent is made up of four major components that define the agent and its capabilities.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="kpgireesh_1-1744861431342.png" style="width: 400px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/16048i6F49B3C5ECD5B069/image-size/medium?v=v2&amp;amp;px=400" role="button" title="kpgireesh_1-1744861431342.png" alt="kpgireesh_1-1744861431342.png" /&gt;&lt;/span&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;&lt;STRONG&gt;Agent/Brain&lt;/STRONG&gt;:&amp;nbsp;&lt;/SPAN&gt;&lt;SPAN&gt;The LLM acts as the brain of an Agent that processes inputs received, understands its inputs and goals to be achieved. It serves as the central decision-making component, instructs tool calls and acts as the command center orchestrating all other components to work towards achieving the goal.&amp;nbsp;&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;&lt;STRONG&gt;Planning&lt;/STRONG&gt;:&amp;nbsp;&lt;/SPAN&gt;&lt;SPAN&gt;Planning is not a component but a key stage in the process where the LLM breaks down larger tasks into smaller more manageable tasks and strategises on next actions. The Planning step involves task decomposition through chain-of-thought reasoning, self-reflection of past actions and evaluation of the effectiveness of ‘strategies’ through internal feedback and before optimising the next best action. Planning is where LLM’s capabilities will be really tested and without robust planning capabilities, an Agent will not be able to achieve its goals.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;STRONG&gt;Memory&lt;/STRONG&gt;:&amp;nbsp;&lt;SPAN&gt;Memory allows Agents to store and retrieve information gathered to review past events and maintain context. Memory consists of two types:&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;UL class="lia-list-style-type-circle"&gt;
&lt;LI&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Short-term memory: Functions as storage for immediate context to maintain continuity during iterations.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Long-term memory: Stores historical information over longer periods of time which can be valuable for future task completion and learnings.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Agent memory can either be a short term memory or a combination of long term and short term memory depending on decision making inputs needed.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/LI&gt;
&lt;/UL&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;STRONG&gt;Tools&lt;/STRONG&gt;:&amp;nbsp;&lt;SPAN&gt;This component integrates external tools and APIs to provide access to information to extend Agent’s capabilities beyond the internal knowledge (LLM knowledge). The tool set should be well designed providing enough tools with thorough description of tools for Agent to understand when and how to use tools. Examples of tools include websearch APIs, API for database access, specialised functions etc..&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&amp;nbsp;&lt;/P&gt;
&lt;H1&gt;&lt;STRONG&gt;Define the Agent&lt;/STRONG&gt;&lt;/H1&gt;
&lt;P&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;While building an Agent, it’s important to establish a use case, purpose and definition of Agent as the first step before diving into the technology aspects. The clarity at this stage is critical for successful outcomes and technology decisions like model selection, tool decisions, and deployment strategies.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Building a case and purpose for Agent&amp;nbsp;&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;The critical success factor for an Agent is selection of the right use case which needs Agent capabilities instead of fitting in Agent as a solution for the problem at hand. Once a use case is identified, we should be able to clearly define the Agent's purpose and goals.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Define the Agent&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Once it's clear what problem Agent will solve, we progress by defining the Agent. The Agent’s definition should include:&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;UL&gt;
&lt;LI&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Problems Agent will solve&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Target audience, expected interactions and response styles&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Fallback strategy for Agent (how to minimise disruptions if Agent fails)&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Autonomy level of Agent (full, partial, human in loop)&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Identification of Tools required by Agent and Tool integration strategy&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Non functional requirements like peak load to be handled, response latency, security considerations&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Agent Evaluation strategy and Evaluation&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Future plans for extending Agent capabilities for new use cases&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/LI&gt;
&lt;/UL&gt;
&lt;H1&gt;&amp;nbsp;&lt;/H1&gt;
&lt;H1&gt;&lt;FONT size="5"&gt;&lt;STRONG&gt;Building the Agent&lt;/STRONG&gt;&lt;/FONT&gt;&lt;/H1&gt;
&lt;P&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Once Agent is defined, we start building the Agent. We will leverage Databricks for building, evaluating and deploying at scale as &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/generative-ai/agent-framework/build-genai-apps#-mosaic-ai-agent-framework" target="_self"&gt;&lt;SPAN&gt;Databricks Mosaic AI Agent Framework&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt;, &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/mlflow/mlflow-tracing" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;MLflow &lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt;and &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/machine-learning/model-serving/manage-serving-endpoints" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;Model Serving&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt; together provides tools and infrastructure to create, deploy and monitor enterprise grade AI Agents with ease. Building Agent involves following key steps&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Feasibility assessment - prototype before building&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Before jumping into building the Agent, the best practice is to do feasibility assessment. This is a fast iterative phase to assess feasibility of Agent as a solution before committing to build, and also identify probable candidate models/tools for building Agent. This is a critical step as in case feasibility is not established, we need to go back to the drawing board to redefine Agent.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Databricks provides out of box Agent prototype capability with &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/generative-ai/agent-framework/ai-playground-agent" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;AI Playground&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt;. Playground provides an UI to select Model, set Prompt and attach Tools to create the Agent prototype with a few clicks and to test combinations of LLM, Tools and Prompts to create variations and assess the feasibility.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="kpgireesh_2-1744861432230.gif" style="width: 400px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/16050i83C021DF96EE818D/image-size/medium?v=v2&amp;amp;px=400" role="button" title="kpgireesh_2-1744861432230.gif" alt="kpgireesh_2-1744861432230.gif" /&gt;&lt;/span&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Setting the AI Agent framework for Agent development&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;An AI Agent framework (like LangGraph) serves as the foundation layer and hence the choice should be made after evaluating available frameworks. Select a framework which has most capabilities supporting the Agent definition we charted for the Agent. Look for Framework which gives a higher level of abstraction and more pre-built modules to reduce complexity and time. If a complex use case needs more custom changes, go for a framework which provides low-level APIs to interact with core components for customisations. Besides, the framework should meet security and scaling requirements (supporting optimisation, distributed processing etc) and should be proven at enterprise level.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Selection of LLM&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;The next step for Agent development is selection of Model/Models, the selection should enable the Agent definition we laid out. In case Agent handles simple to complex tasks, consider multi-model strategy to have basic models to handle simple tasks and superior reasoning capable models to handle complex tasks (use dynamic routing to direct inputs to appropriate models), basically match model capability to match task, performance and deployment requirements. If Agent handles specialised tasks, prefer LLM which score high on those task specific benchmarks (like HumanEval for Python coding if Agent needs to perform Python coding). We need an iterative approach to try and select a set of best suited models&amp;nbsp; and keep options to try new models during development and after deployment. LLM selection should balance meeting optimal performance, cost efficiency and meeting scalability and other operational requirements. Leverage Databricks &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/generative-ai/agent-framework/ai-playground-agent" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;AI Playground&lt;/SPAN&gt;&lt;/A&gt;&lt;/FONT&gt;&lt;SPAN&gt;&lt;FONT size="4"&gt; for quick experiments with models for candidate model selection.&lt;/FONT&gt;&amp;nbsp;&lt;/SPAN&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Building Tools&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;This step implements the tools within Agent according to tools and tool integration strategy decided during Agent definition. Here are options we can explore&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-60px"&gt;&lt;SPAN&gt;&lt;STRONG&gt;Databricks Unity Catalog (UC) functions as tools&lt;/STRONG&gt;&lt;/SPAN&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;:&amp;nbsp;&lt;/SPAN&gt;&lt;/FONT&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Databricks Unity Catalog (UC) &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-create-sql-function" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;functions&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt; are an effective way to create tools. Databricks provides &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/generative-ai/agent-framework/create-custom-tool#what-is-ucfunctiontoolkit" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;UCFunctionToolkit&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt; which simplifies creating and using functions as tools abstracting complexities and provides a uniform and consistent method for defining&lt;/SPAN&gt; &lt;SPAN&gt;and using tools. Further, it provides auto tracing to track tool performance and retrieved information which are valuable for building observability.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-60px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;&lt;STRONG&gt;Add third party Tools&lt;/STRONG&gt;:&amp;nbsp;&lt;/SPAN&gt;&lt;SPAN&gt;Third party tools can be easily &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/generative-ai/agent-framework/external-connection-tools#connect-ai-agent-tools-to-external-services" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;integrated&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt;, these tools can be API based access to tools (eg: to create a new order, access to create order API) , access to documents (eg: to access to refund policy documentation to confirm refund eligibility), third party API access (eg: access to Google Map API for locations or access to Salesforce API to interact with CRM data)&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Providing memory&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;The memory component needs to be designed to meet the demand of information context and persistence level needed for Agent to function (example: track last six interactions to maintain context in short term memory, retain user preference from past one year in long term memory). Store only relevant information and only for required duration to optimize efficiency, reduce latency and cost. Reduce PII and sensitive data dependency and if unavoidable, comply with compliance requirements of storing PII data. Prioritise latency for real-time tasks (use in memory cache like Redis) and for high-volume/Long-term go for scalable multimodal databases like Databricks Brickstore. Databricks Mosaic Agent Framework provides seamless and secure integration for building Agent memory.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Building the prompts&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Providing effective prompts is crucial for directing LLM towards Agents goals. Prompt is an area which needs a lot of iterations to get to prompt which can maximise Agents performance. Prompt should include Role and Context, Task Description, Specific and step by step instructions for Agent, mention sources of information, define output format expected (length, format, style etc..)&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;MLflow &lt;/SPAN&gt;&lt;A href="https://mlflow.org/docs/latest/prompts" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;Prompt Management &lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt;is a powerful tool for prompt engineering and management providing features like Version control, Centralised prompt registry for Reusability, Change management and Collaboration, Enabling flexible deployment isolating prompt from application code, Facilitating prompt comparison and tracking prompts used by models and apps. It’s highly recommended to use MLflow for centralised prompt management for building Agents.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Building the Agent - putting it all together&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;We build the agent integrating all components using the Agent Framework we selected. The Framework takes care of orchestrating, tool integration, memory management and LLM by bringing together all components to function as Agent.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;H1&gt;&amp;nbsp;&lt;/H1&gt;
&lt;H1&gt;&lt;FONT size="5"&gt;&lt;STRONG&gt;Evaluation of Agent&lt;/STRONG&gt;&lt;/FONT&gt;&lt;/H1&gt;
&lt;P&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Once Agent is built, the Agent needs to be evaluated to ensure Agent definition charted for Agents works which include functional, non-functional and behavior aspects. The evaluation is an iterative process and hence the entire solution should be flexible to adapt and change as needed based on evaluation results. The recommended critical areas for evaluation are Accuracy, Stability (consistency and reliability), Cost, Latency, Response Style and Security. It is critical to involve business users, product team, sales team and behaviour experts (validate Agent behaviour and response) during the evaluation process.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;The evaluation is made simple and comprehensive by &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/generative-ai/agent-evaluation/" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;Mosaic AI Agent evaluation&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt; providing advanced evaluation techniques using SDK and UI to define evaluation criteria specific to your business needs. Features like &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/generative-ai/agent-evaluation/llm-judge-reference" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;Built-in AI judges&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt;, &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/generative-ai/agent-evaluation/advanced-agent-eval" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;Customize AI judges&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt;, &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/generative-ai/agent-evaluation/custom-metrics" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;Custom metrics&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt;, review app for human reviews cover all qualitative and quantitative metrics that are used to evaluate quality .&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="kpgireesh_3-1744861431893.gif" style="width: 400px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/16051i6E1413301C78A773/image-size/medium?v=v2&amp;amp;px=400" role="button" title="kpgireesh_3-1744861431893.gif" alt="kpgireesh_3-1744861431893.gif" /&gt;&lt;/span&gt;&lt;/P&gt;
&lt;H1&gt;&amp;nbsp;&lt;/H1&gt;
&lt;H1&gt;&lt;FONT size="5"&gt;&lt;STRONG&gt;Productionising Agent&lt;/STRONG&gt;&lt;/FONT&gt;&lt;/H1&gt;
&lt;P&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Once evaluation is successful, the Agent can be moved to production. This phase aims at building infrastructure to serve Agent, ensure security and provide governance&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Model Serving&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Productionising Agents require an infrastructure which can scale at optimal cost, easy to maintain/operate and easy to deploy model.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Databricks &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/machine-learning/model-serving" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;Model Serving&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt; (a fully managed serverless infrastructure) is an ideal choice for running Agents. Databricks model serving has simplified deployment with few lines of code with automatic scaling and optimisation. Besides providing multi-model serving capabilities and deep integration with data platforms&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Security and Governance&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;AI agents comprise several interconnected components that each present distinct security considerations and security challenges unlike traditional software. A security-first architecture establishes safeguards at every layer of the AI agent system by having least privilege for all agent components and interactions (like giving only read access to certain data in a table and not full read/delete/update etc.)&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Databricks provides robust capabilities for security and governance through the Mosaic AI Gateway, Unity Catalog and Databricks AI Security Framework (DASF).&amp;nbsp;&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Mosaic AI Gateway, provides centralized permissions, guardrails, rate limits for model access,&amp;nbsp; payload logging which ensure compliance, security, auditing and monitoring compliance, besides providing traffic fallback mechanisms to ensure system reliability. Unity Catalog provide Data governance, secure integration with external APIs and enterprise tools and DASF provides comprehensive framework for managing AI risks, aligned with recognized standards such as MITRE ATLAS and NIST&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;H1&gt;&amp;nbsp;&lt;/H1&gt;
&lt;H1&gt;&lt;FONT size="5"&gt;&lt;STRONG&gt;Agent monitoring and observability (operating the Agent)&lt;/STRONG&gt;&lt;/FONT&gt;&lt;/H1&gt;
&lt;P&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Agents being autonomous and complex, implementing robust monitoring and observability systems is critical for ensuring reliability and performance. Without proper observability Agents become "black boxes" that are difficult to debug and trust. Further data from monitoring and observability can be used for online evaluation of Agents for identification of issues and for future improvements.This is a rapidly evolving area and standardized protocols like OpenTelemetry (OTel) and OpenLLMetry are trying to unify observability approaches&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Agent Monitoring&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;We should clearly define KPI which needs to be monitored for building the monitoring system (like accuracy &amp;gt; 95%, latency &amp;lt; 500ms) and implement logging systems which can capture data for these KPI’s. The best practice is to integrate monitoring with organisation central monitoring and alert mechanism to have a centralized visibility and response system.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;A href="https://docs.databricks.com/aws/en/generative-ai/agent-evaluation" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;Mosaic AI Agent Monitoring&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt; provides operational and quality metrics with flexibility to configure to get KPI’s operations needs.It also provides evaluation judges (built-in and custom) to track advanced quality metrics.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="5"&gt;&lt;SPAN&gt;Agent Observability&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Observability is to understand what's happening inside Agents by examining external signals like logs, metrics, and traces to provide deeper visibility into internal working of Agent like Actions and tool usage patterns, Model calls and responses etc..&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;FONT size="4"&gt;&lt;A href="https://docs.databricks.com/gcp/en/mlflow/mlflow-tracing" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;Databricks Agent observability&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt; with MLflow tracing enables automatic and manual tracing to create complete tracing of Agent in real-time.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P class="lia-indent-padding-left-30px"&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="kpgireesh_4-1744861432074.gif" style="width: 400px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/16052iDF3112899CD04107/image-size/medium?v=v2&amp;amp;px=400" role="button" title="kpgireesh_4-1744861432074.gif" alt="kpgireesh_4-1744861432074.gif" /&gt;&lt;/span&gt;&lt;/P&gt;
&lt;H1&gt;&amp;nbsp;&lt;/H1&gt;
&lt;H1&gt;&lt;FONT size="5"&gt;&lt;STRONG&gt;Conclusion&lt;/STRONG&gt;&lt;/FONT&gt;&lt;/H1&gt;
&lt;P&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;Building AI Agents is not just about leveraging cutting-edge technology—it's about creating intelligent systems that can adapt to dynamic business needs and deliver measurable value. From defining the Agent's purpose to deploying it securely at scale, every step in this journey requires careful planning and execution. With tools like Databricks Mosaic AI Agent Framework and MLflow, the complexities of development and deployment are greatly simplified, enabling organizations to focus on innovation rather than infrastructure.&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;
&lt;P&gt;&lt;FONT size="4"&gt;&lt;SPAN&gt;As businesses continue to embrace AI-driven solutions, AI Agents stand out as transformative tools capable of solving real-world challenges with autonomy and precision. Whether you're looking to streamline operations or enhance customer experiences, this guide equips you with the foundational knowledge to build robust AI Agents tailored to your specific use cases. The future of AI is here—are you ready to harness its full potential with Databricks?&lt;/SPAN&gt;&lt;/FONT&gt;&lt;/P&gt;</description>
    <pubDate>Mon, 21 Apr 2025 16:45:00 GMT</pubDate>
    <dc:creator>kpgireesh</dc:creator>
    <dc:date>2025-04-21T16:45:00Z</dc:date>
    <item>
      <title>Building Intelligent AI Agents: The Complete Guide From Blueprint to Enterprise Deployment</title>
      <link>https://community.databricks.com/t5/technical-blog/building-intelligent-ai-agents-the-complete-guide-from-blueprint/ba-p/115708</link>
      <description>&lt;P class="my-0"&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="Screenshot 2025-04-17 at 9.27.19 AM.png" style="width: 999px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/16053iA5C5BC998592B209/image-size/large?v=v2&amp;amp;px=999" role="button" title="Screenshot 2025-04-17 at 9.27.19 AM.png" alt="Screenshot 2025-04-17 at 9.27.19 AM.png" /&gt;&lt;/span&gt;Ready to transform your enterprise with the next generation of AI? Imagine intelligent agents that don’t just answer questions, but autonomously plan, reason, and act—tailored precisely to your business needs. This definitive guide takes you behind the scenes of building robust AI Agents, from blueprint to enterprise deployment, using powerful frameworks like Databricks Mosaic AI.&lt;/P&gt;
&lt;P class="my-0"&gt;Whether you’re a developer eager to master architecture, a data scientist seeking best practices for model selection, or a business leader looking to deploy secure, scalable AI solutions, this article delivers actionable insights for every role. Discover how to design agents with advanced memory, integrate specialized tools, and ensure your solutions are reliable, secure, and future-proof.&lt;/P&gt;
&lt;P class="my-0"&gt;Unlock the secrets to evaluating, deploying, and monitoring AI Agents that can revolutionize customer service, research, and decision-making in your organization. Dive in to learn strategies you can apply immediately—and stay ahead as AI reshapes the enterprise landscape. The future of intelligent automation starts here. Don’t miss out—your blueprint to building enterprise-grade Agent AI awaits&lt;/P&gt;</description>
      <pubDate>Mon, 21 Apr 2025 16:45:00 GMT</pubDate>
      <guid>https://community.databricks.com/t5/technical-blog/building-intelligent-ai-agents-the-complete-guide-from-blueprint/ba-p/115708</guid>
      <dc:creator>kpgireesh</dc:creator>
      <dc:date>2025-04-21T16:45:00Z</dc:date>
    </item>
  </channel>
</rss>

