<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: BUG: Agent deployment fails with PERMISSION_DENIED for Lakebase dependency when created via data in Lakebase Discussions</title>
    <link>https://community.databricks.com/t5/lakebase-discussions/bug-agent-deployment-fails-with-permission-denied-for-lakebase/m-p/146616#M3</link>
    <description>&lt;P&gt;Thanks&amp;nbsp;for your answer&amp;nbsp;&lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/202980"&gt;@pradeep_singh&lt;/a&gt;,&lt;/P&gt;&lt;P&gt;We are using MLFlow version 3.8.1&lt;/P&gt;&lt;P&gt;Update: We were told by a Databricks employee that this is well-known issue and that currently only workspace admins can properly passthrough credentials while agent endpoint creation. We tested again making the endpoint creator a workspace admin and it worked.&amp;nbsp;&lt;/P&gt;&lt;P&gt;P.S. Another bug we found is that even after passing the parameter&amp;nbsp;&lt;/P&gt;&lt;DIV&gt;&lt;SPAN&gt;scale_to_zero_enabled&lt;/SPAN&gt;&lt;SPAN&gt;=&lt;/SPAN&gt;&lt;SPAN&gt;&lt;SPAN&gt;True to&amp;nbsp;&lt;/SPAN&gt;&lt;/SPAN&gt;&lt;SPAN&gt;agents&lt;/SPAN&gt;&lt;SPAN&gt;.&lt;/SPAN&gt;&lt;SPAN&gt;deploy(), the endpoint is not marked correctly to scale to zero. We had to manually change that through the UI.&lt;/SPAN&gt;&lt;/DIV&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
    <pubDate>Mon, 02 Feb 2026 18:24:51 GMT</pubDate>
    <dc:creator>andres920310</dc:creator>
    <dc:date>2026-02-02T18:24:51Z</dc:date>
    <item>
      <title>BUG: Agent deployment fails with PERMISSION_DENIED for Lakebase dependency when created via databric</title>
      <link>https://community.databricks.com/t5/lakebase-discussions/bug-agent-deployment-fails-with-permission-denied-for-lakebase/m-p/146047#M1</link>
      <description>&lt;H2&gt;&lt;STRONG&gt;Summary&lt;/STRONG&gt;&lt;/H2&gt;&lt;P&gt;We are encountering a &lt;STRONG&gt;PERMISSION_DENIED error when deploying a Databricks Agent that uses Lakebase for agent memory&lt;/STRONG&gt;, even though the endpoint creator has all documented permissions.&lt;/P&gt;&lt;P&gt;The failure happens during &lt;STRONG&gt;serving endpoint creation&lt;/STRONG&gt;, which is triggered from a &lt;STRONG&gt;job task running a notebook&lt;/STRONG&gt; that uses the &lt;STRONG&gt;databricks-agents&lt;/STRONG&gt; library.&lt;/P&gt;&lt;P&gt;Based on the documentation and our permission setup, this deployment &lt;STRONG&gt;should succeed&lt;/STRONG&gt;, but it fails when Databricks attempts to grant permissions to the served entity’s service principal.&lt;/P&gt;&lt;P&gt;We believe this is a &lt;STRONG&gt;Databricks bug related to Lakebase permission handling during agent deployment&lt;/STRONG&gt;.&lt;/P&gt;&lt;HR /&gt;&lt;H2&gt;&lt;STRONG&gt;Environment&lt;/STRONG&gt;&lt;/H2&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Cloud: AWS&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Deployment mechanism:&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Databricks Asset Bundles (DABs)&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Job task running a notebook that uses databricks-agents to create the serving endpoint&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Features involved:&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Databricks Agents Framework&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Lakebase (used for agent memory)&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;MLflow model logging with resource dependencies&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;HR /&gt;&lt;H2&gt;&lt;STRONG&gt;What We’re Doing&lt;/STRONG&gt;&lt;/H2&gt;&lt;OL&gt;&lt;LI&gt;&lt;P&gt;Create a &lt;STRONG&gt;Lakebase instance&lt;/STRONG&gt; to be used as agent memory&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Log an agent model using MLflow, explicitly declaring the Lakebase dependency:&lt;/P&gt;&lt;DIV class=""&gt;&lt;DIV class=""&gt;&lt;DIV class=""&gt;&lt;DIV class=""&gt;&amp;nbsp;&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;DIV class=""&gt;&lt;SPAN&gt;from mlflow.models.resources import DatabricksLakebase&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV class=""&gt;&amp;nbsp;&lt;/DIV&gt;&lt;DIV class=""&gt;&lt;SPAN&gt;mlflow.pyfunc.log_model(&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV class=""&gt;&lt;SPAN&gt;...&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV class=""&gt;&lt;SPAN&gt;resources=[&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV class=""&gt;&lt;SPAN&gt;DatabricksLakebase(database_instance_name=&lt;SPAN class=""&gt;"agent-memory",&lt;/SPAN&gt;&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV class=""&gt;&lt;SPAN&gt;] ) &lt;/SPAN&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Deploy the agent by:&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Executing a &lt;STRONG&gt;job task c&lt;/STRONG&gt;alling the &lt;STRONG&gt;databricks-agents&lt;/STRONG&gt; library to create or update a serving endpoint&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/LI&gt;&lt;/OL&gt;&lt;HR /&gt;&lt;H2&gt;&lt;STRONG&gt;Permissions Setup (Confirmed)&lt;/STRONG&gt;&lt;/H2&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;The user executing the job:&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Created the Lakebase instance&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Is the endpoint creator&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Has the databricks_superuser role on the Lakebase database (&lt;SPAN&gt;This aligns with the documented permission requirements for automatic authentication passthrough:&lt;/SPAN&gt;&lt;/P&gt;&lt;A class="" href="https://docs.databricks.com/aws/en/generative-ai/agent-framework/agent-authentication?language=Vector+Search+Retriever+Tool#supported-resources-for-automatic-authentication-passthrough" target="_new" rel="noopener"&gt;https://docs.databricks.com/aws/en/generative-ai/agent-framework/agent-authentication?language=Vector+Search+Retriever+Tool#supported-resources-for-automatic-authentication-passthrough)&lt;/A&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Has explicit CAN_MANAGE permissions on:&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;The Lakebase instance&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;All other resources created via the DAB&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;HR /&gt;&lt;H2&gt;&lt;STRONG&gt;Expected Behavior&lt;/STRONG&gt;&lt;/H2&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;The serving endpoint should be created successfully&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Databricks should be able to grant the served entity’s &lt;STRONG&gt;service principal&lt;/STRONG&gt; access to the Lakebase dependency automatically&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;HR /&gt;&lt;H2&gt;&lt;STRONG&gt;Actual Behavior&lt;/STRONG&gt;&lt;/H2&gt;&lt;P&gt;The deployment fails during &lt;STRONG&gt;served entity creation&lt;/STRONG&gt;, with the following error:&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;DIV class=""&gt;&lt;DIV class=""&gt;&lt;DIV class=""&gt;&lt;DIV class=""&gt;&amp;nbsp;&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;DIV class=""&gt;&lt;SPAN&gt;Endpoint update failed Failed &lt;SPAN class=""&gt;to&lt;/SPAN&gt; deploy agent_model_1: Pre-deployment setup &lt;SPAN class=""&gt;for&lt;/SPAN&gt; served entity &lt;SPAN class=""&gt;with&lt;/SPAN&gt; name &lt;SPAN class=""&gt;'agent_model_1'&lt;/SPAN&gt; &lt;SPAN class=""&gt;and&lt;/SPAN&gt; version &lt;SPAN class=""&gt;'1' failed.&lt;/SPAN&gt; &lt;SPAN class=""&gt;Error:&lt;/SPAN&gt; Served entity service creation failed. This often happens due &lt;SPAN class=""&gt;to&lt;/SPAN&gt; failure &lt;SPAN class=""&gt;to&lt;/SPAN&gt; grant the service principal associated &lt;SPAN class=""&gt;with&lt;/SPAN&gt; the served entity permission &lt;SPAN class=""&gt;to&lt;/SPAN&gt; access one &lt;SPAN class=""&gt;or&lt;/SPAN&gt; more Databricks product resources. &lt;SPAN class=""&gt;Error:&lt;/SPAN&gt; PERMISSION_DENIED: Failed &lt;SPAN class=""&gt;to&lt;/SPAN&gt; change permissions &lt;SPAN class=""&gt;for&lt;/SPAN&gt; SP &lt;SPAN class=""&gt;3e86&lt;/SPAN&gt;aa94-&lt;SPAN class=""&gt;20e8&lt;/SPAN&gt;-&lt;SPAN class=""&gt;4&lt;/SPAN&gt;a99-aa48-&lt;SPAN class=""&gt;7e4&lt;/SPAN&gt;ae9fb895f. &lt;SPAN class=""&gt;Reason:&lt;/SPAN&gt; &lt;SPAN class=""&gt;PERMISSION_DENIED:&lt;/SPAN&gt; Endpoint creator doesn&lt;SPAN class=""&gt;'t have permission to access dependency type: LAKEBASE&lt;/SPAN&gt; &lt;SPAN class=""&gt;with&lt;/SPAN&gt; name: agent-memory&lt;/SPAN&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;HR /&gt;&lt;H2&gt;&lt;STRONG&gt;Why We Believe This Is a Bug&lt;/STRONG&gt;&lt;/H2&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;The &lt;STRONG&gt;same user&lt;/STRONG&gt;:&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Creates the Lakebase&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Logs the model&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Executes the job&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Creates the serving endpoint&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;The Lakebase dependency is &lt;STRONG&gt;explicitly declared at model logging time&lt;/STRONG&gt; using mlflow.models.resources.DatabricksLakebase&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;The user has:&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;databricks_superuser role&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Explicit CAN_MANAGE permissions&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;The error occurs &lt;STRONG&gt;when Databricks internally attempts to grant permissions to the served entity’s service principal&lt;/STRONG&gt;&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;The error message claims the endpoint creator lacks permission, which contradicts:&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Actual permissions&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Documented requirements&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;P&gt;This suggests a bug in one of the following areas:&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Permission validation for &lt;STRONG&gt;Lakebase dependencies&lt;/STRONG&gt; during agent deployment&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Service principal permission propagation for Lakebase&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Handling of Lakebase as a dependency type in the Agents framework&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;HR /&gt;&lt;H2&gt;&lt;STRONG&gt;Request&lt;/STRONG&gt;&lt;/H2&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Can the Databricks team confirm whether:&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;This is a known issue with Agents + Lakebase?&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;There are additional (currently undocumented) permissions required?&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;If this is a bug, we would appreciate help escalating this to the relevant engineering team.&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;P&gt;We’re happy to provide workspace details or a full repro privately if needed.&lt;/P&gt;</description>
      <pubDate>Fri, 30 Jan 2026 12:59:18 GMT</pubDate>
      <guid>https://community.databricks.com/t5/lakebase-discussions/bug-agent-deployment-fails-with-permission-denied-for-lakebase/m-p/146047#M1</guid>
      <dc:creator>andres920310</dc:creator>
      <dc:date>2026-01-30T12:59:18Z</dc:date>
    </item>
    <item>
      <title>Re: BUG: Agent deployment fails with PERMISSION_DENIED for Lakebase dependency when created via data</title>
      <link>https://community.databricks.com/t5/lakebase-discussions/bug-agent-deployment-fails-with-permission-denied-for-lakebase/m-p/146425#M2</link>
      <description>&lt;P&gt;Can you check the MLFlow version you are using .&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="pradeep_singh_0-1769958322721.png" style="width: 400px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/23548iB1AC7D2DE39B5E93/image-size/medium?v=v2&amp;amp;px=400" role="button" title="pradeep_singh_0-1769958322721.png" alt="pradeep_singh_0-1769958322721.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;A href="https://docs.databricks.com/aws/en/generative-ai/agent-framework/agent-authentication#supported-resources-for-automatic-authentication-passthrough" target="_blank"&gt;https://docs.databricks.com/aws/en/generative-ai/agent-framework/agent-authentication#supported-resources-for-automatic-authentication-passthrough&lt;/A&gt;&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Sun, 01 Feb 2026 15:06:01 GMT</pubDate>
      <guid>https://community.databricks.com/t5/lakebase-discussions/bug-agent-deployment-fails-with-permission-denied-for-lakebase/m-p/146425#M2</guid>
      <dc:creator>pradeep_singh</dc:creator>
      <dc:date>2026-02-01T15:06:01Z</dc:date>
    </item>
    <item>
      <title>Re: BUG: Agent deployment fails with PERMISSION_DENIED for Lakebase dependency when created via data</title>
      <link>https://community.databricks.com/t5/lakebase-discussions/bug-agent-deployment-fails-with-permission-denied-for-lakebase/m-p/146616#M3</link>
      <description>&lt;P&gt;Thanks&amp;nbsp;for your answer&amp;nbsp;&lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/202980"&gt;@pradeep_singh&lt;/a&gt;,&lt;/P&gt;&lt;P&gt;We are using MLFlow version 3.8.1&lt;/P&gt;&lt;P&gt;Update: We were told by a Databricks employee that this is well-known issue and that currently only workspace admins can properly passthrough credentials while agent endpoint creation. We tested again making the endpoint creator a workspace admin and it worked.&amp;nbsp;&lt;/P&gt;&lt;P&gt;P.S. Another bug we found is that even after passing the parameter&amp;nbsp;&lt;/P&gt;&lt;DIV&gt;&lt;SPAN&gt;scale_to_zero_enabled&lt;/SPAN&gt;&lt;SPAN&gt;=&lt;/SPAN&gt;&lt;SPAN&gt;&lt;SPAN&gt;True to&amp;nbsp;&lt;/SPAN&gt;&lt;/SPAN&gt;&lt;SPAN&gt;agents&lt;/SPAN&gt;&lt;SPAN&gt;.&lt;/SPAN&gt;&lt;SPAN&gt;deploy(), the endpoint is not marked correctly to scale to zero. We had to manually change that through the UI.&lt;/SPAN&gt;&lt;/DIV&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Mon, 02 Feb 2026 18:24:51 GMT</pubDate>
      <guid>https://community.databricks.com/t5/lakebase-discussions/bug-agent-deployment-fails-with-permission-denied-for-lakebase/m-p/146616#M3</guid>
      <dc:creator>andres920310</dc:creator>
      <dc:date>2026-02-02T18:24:51Z</dc:date>
    </item>
  </channel>
</rss>

