<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>All Generative AI posts</title>
    <link>https://community.databricks.com/t5/generative-ai/bd-p/GenAI-Insight-Hub</link>
    <description>All Generative AI posts</description>
    <pubDate>Tue, 17 Nov 2026 01:24:39 GMT</pubDate>
    <dc:creator>GenAI-Insight-Hub</dc:creator>
    <dc:date>2026-11-17T01:24:39Z</dc:date>
    <item>
      <title>Re: Protecting Your Company Data When Integrating Coding Agents through Unity AI Gateway</title>
      <link>https://community.databricks.com/t5/generative-ai/protecting-your-company-data-when-integrating-coding-agents/m-p/176570#M2272</link>
      <description>&lt;DIV class=""&gt;&lt;P class=""&gt;The SQL policy for blocking private keys is a strong starting point. To make these rules easier to manage, it helps to clarify ownership: defining what counts as an unapproved credential or sensitive token often involves both security and platform engineering.&lt;/P&gt;&lt;P class=""&gt;For a scalable approach, I’d recommend addressing this from two angles. First, complement runtime gateway controls with automated scans across codebases and environments. This can help identify and resolve risky secret handling at the source, before an agent encounters it.&lt;/P&gt;&lt;P class=""&gt;Second, custom SQL policies at the gateway benefit from a reliable testing process. Historical gateway run data can serve as an offline evaluation suite. Before updating a policy, such as adding a pattern to block, you can test it against past runs to assess whether it would interfere with valid developer tasks or fail to detect sensitive data. This makes policy updates more predictable and helps teams improve protections with confidence.&lt;/P&gt;&lt;/DIV&gt;</description>
      <pubDate>Mon, 16 Nov 2026 16:00:30 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/protecting-your-company-data-when-integrating-coding-agents/m-p/176570#M2272</guid>
      <dc:creator>slowder</dc:creator>
      <dc:date>2026-11-16T16:00:30Z</dc:date>
    </item>
    <item>
      <title>Protecting Your Company Data When Integrating Coding Agents through Unity AI Gateway</title>
      <link>https://community.databricks.com/t5/generative-ai/protecting-your-company-data-when-integrating-coding-agents/m-p/176397#M2271</link>
      <description>&lt;P&gt;Coding agents like Claude Code, Codex, OpenCode, or PI, started through the ug launcher, and integrated with foundation models through Unity AI Gateway can run with whatever Databricks permissions the developer already has through MCPs that is native on Databricks such as AI Search MCP(s), or Genie MCP(s).&lt;/P&gt;&lt;P&gt;Basically this introduce risk to the company if you are trying to protect the data before the model for whatever the reason is AI governance, law, etc, ...&lt;/P&gt;&lt;P&gt;We have been adopting protective mechanism in layers, where each one catches what the one above it misses.&lt;/P&gt;&lt;P&gt;- The first layer is what you already have. Since the agent acts as the user, Unity Catalog row filters, column masks, ABAC tag policies and plain least-privilege grants still apply to whatever it reads. But usually, people have multiple permissions and we want to protect the coding agent from reading the PII from those who have such permissions.&lt;/P&gt;&lt;P&gt;- The second layer is Sensitive Data Detection policy on the model service, you can turn this on on the UI and set it at rank 1, with tags email_address and credit_card. The model gets a token such as [US_SSN] instead of the value. Everything else just allows or denies. Some limitations that we have experience, the REDACT mechanism can be sometime to strict and remove important information from the coding agent such as AWS account number, Databricks workspace id when it is trying to perform task on such environment and break the workflow.&lt;/P&gt;&lt;P&gt;- The third layer is for what the built-ins miss or you can replace it completely to match some of your workflow. A custom SQL policy is a function that takes the event and returns a decision. The body is converted to a small SQL subset, so no regexp, just things like CONTAINS, LIKE and STARTSWITH.&amp;nbsp;&lt;/P&gt;&lt;PRE&gt;CREATE OR REPLACE FUNCTION main.governance.block_private_keys(
  event VARIANT
)
RETURNS VARIANT
LANGUAGE SQL
RETURN
  CASE
    WHEN event:type::string = 'request'
      AND CONTAINS(event:context.message::string, 'PRIVATE KEY-----')
    THEN to_variant_object(named_struct('result', 'DENY', 'reason', 'Prompt contains a private key.'))
    ELSE to_variant_object(named_struct('result', 'ALLOW', 'reason', ''))
  END;&lt;/PRE&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Sat, 14 Nov 2026 10:30:49 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/protecting-your-company-data-when-integrating-coding-agents/m-p/176397#M2271</guid>
      <dc:creator>DoTA</dc:creator>
      <dc:date>2026-11-14T10:30:49Z</dc:date>
    </item>
    <item>
      <title>Re: Dashboards dual axis</title>
      <link>https://community.databricks.com/t5/generative-ai/dashboards-dual-axis/m-p/176255#M2270</link>
      <description>&lt;P&gt;&lt;STRONG&gt;Hi &lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/270858"&gt;@hhaaland&lt;/a&gt;,&lt;/STRONG&gt;&lt;/P&gt;
&lt;P&gt;&lt;STRONG&gt;Current documentation on the&amp;nbsp;AI/BI dashboards with dual y-axis states that axis range has three options:&lt;/STRONG&gt;&lt;/P&gt;
&lt;OL&gt;
&lt;LI&gt;&lt;STRONG&gt;Default &lt;/STRONG&gt;("Use an automatic range")&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;Per facet,&lt;/STRONG&gt; which only applies to faceted charts&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;Custom&lt;/STRONG&gt; ("Set fixed Minimum and Maximum values")&lt;/LI&gt;
&lt;/OL&gt;
&lt;UL&gt;
&lt;LI&gt;There's no "include zero" toggle, and the docs don't say whether you can fill in only Minimum and leave Maximum blank.
&lt;UL&gt;
&lt;LI&gt;Log (Symmetric) doesn't help, and internally there's a known report of log axes still starting at&amp;nbsp;0.&lt;/LI&gt;
&lt;LI&gt;Custom Viz uses Vega-Lite, and the docs' own example uses "scale": {"zero": false}.&lt;/LI&gt;
&lt;/UL&gt;
&lt;/LI&gt;
&lt;/UL&gt;
&lt;P&gt;So, I don't think Default can do better on its own. The automatic range starts at zero, and the only other options are Per facet (for faceted charts) and Custom, which fixes both ends. A few ways around it:&lt;/P&gt;
&lt;OL&gt;
&lt;LI&gt;&lt;STRONG&gt;Custom Viz (currently in Public Preview).&lt;/STRONG&gt; Under Advanced visualisation, pick Custom Viz, add your date and two measures in Fields, and paste a Vega-Lite spec. "zero": false lets each axis fit its own data, and "resolve" keeps the two axes independent, so it still rescales when the data changes:
&lt;OL&gt;
&lt;LI&gt;&lt;LI-CODE lang="javascript"&gt;{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "data": {"name": "databricks_query"},
  "width": "container",
  "height": "container",
  "config": {"autosize": {"type": "fit", "contains": "padding"}},
  "layer": [
    {
      "mark": {"type": "line", "color": "#00A972"},
      "encoding": {
        "x": {"field": "date", "type": "temporal"},
        "y": {"field": "series_a", "type": "quantitative",
              "scale": {"zero": false}, "axis": {"title": "Series A"}}
      }
    },
    {
      "mark": {"type": "line", "color": "#FF3621"},
      "encoding": {
        "x": {"field": "date", "type": "temporal"},
        "y": {"field": "series_b", "type": "quantitative",
              "scale": {"zero": false}, "axis": {"title": "Series B", "orient": "right"}}
      }
    }
  ],
  "resolve": {"scale": {"y": "independent"}}
}
&lt;/LI-CODE&gt;&lt;/LI&gt;
&lt;LI&gt;Replace date, series_a and series_b with the Name of each field in the Fields section. Keep "data": {"name": "databricks_query"} as it is: that's how the chart picks up your dataset. More examples are in the docs: &lt;A href="https://docs.databricks.com/aws/en/dashboards/manage/visualizations/custom-visualizations" target="_blank"&gt;Custom visualizations&lt;/A&gt;.&lt;/LI&gt;
&lt;/OL&gt;
&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;If you stay with the built-in chart,&lt;/STRONG&gt; set a Custom range on each axis separately (kebab menu next to &lt;STRONG&gt;Left Y axis&lt;/STRONG&gt; / &lt;STRONG&gt;Right Y axis&lt;/STRONG&gt;). Leave a margin above and below your usual values so new data doesn't fall outside the range.&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;Check whether you need two axes.&lt;/STRONG&gt; In your screenshot both lines sit around 0.45. If they're the same unit, one axis makes the gap between them easier to read.&lt;/LI&gt;
&lt;/OL&gt;
&lt;P&gt;&lt;SPAN&gt;A native "don't start at zero" option would be the cleaner fix, so it's worth raising as an idea&amp;nbsp;&lt;/SPAN&gt;through the&lt;A href="https://www.databricks.com/feedback" target="_self"&gt; Ideas Portal&lt;/A&gt; too.&lt;/P&gt;</description>
      <pubDate>Thu, 12 Nov 2026 13:36:53 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/dashboards-dual-axis/m-p/176255#M2270</guid>
      <dc:creator>Valeria_Koz_DBX</dc:creator>
      <dc:date>2026-11-12T13:36:53Z</dc:date>
    </item>
    <item>
      <title>Re: How to size your Genie bill (Genie Agent, Genie Code) before the free period ends (31 January 20</title>
      <link>https://community.databricks.com/t5/generative-ai/how-to-size-your-genie-bill-genie-agent-genie-code-before-the/m-p/176251#M2269</link>
      <description>&lt;P&gt;Thanks&amp;nbsp;&lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/125801"&gt;@DoTA&lt;/a&gt;&amp;nbsp; this is exactly the kind of review a cost post needs, and all three points were right. I've updated the post:&lt;BR /&gt;&lt;STRONG&gt;&lt;BR /&gt;1. NULL price:&lt;/STRONG&gt; step 4 no longer borrows a billed SKU. The step 3 query multiplies by your own price per DBU, so a free-only workspace can't come back empty.&lt;BR /&gt;&lt;STRONG&gt;2. Split by surface:&lt;/STRONG&gt; the query now returns Genie Code, Genie One and Genie Agents DBUs separately, and takes the 150 DBU allowance off once per person across all of them. Service principals get their own row, with no allowance.&lt;BR /&gt;&lt;STRONG&gt;3. Rendering: the SQL is now a proper code block you can copy.&lt;/STRONG&gt;&lt;BR /&gt;While re-testing it I also made it default to the last complete month, so it returns rows without editing the dates. Thanks again for reading it so carefully. If anyone runs it and compares it with a real February bill, I'd love to hear how close it gets.&lt;/P&gt;</description>
      <pubDate>Thu, 12 Nov 2026 12:20:10 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/how-to-size-your-genie-bill-genie-agent-genie-code-before-the/m-p/176251#M2269</guid>
      <dc:creator>Valeria_Koz_DBX</dc:creator>
      <dc:date>2026-11-12T12:20:10Z</dc:date>
    </item>
    <item>
      <title>How Do I Choose the Right AI Development Services for My Business?</title>
      <link>https://community.databricks.com/t5/generative-ai/how-do-i-choose-the-right-ai-development-services-for-my/m-p/176197#M2268</link>
      <description>&lt;P&gt;I'm exploring AI development services to help businesses automate repetitive tasks, improve operational efficiency, and deliver better customer experiences. With so many companies offering custom AI solutions, machine learning, AI chatbots, predictive analytics, and intelligent automation, choosing the right AI development partner can be challenging.&lt;/P&gt;&lt;P&gt;While researching different providers, I came across App Maisters, a software development company offering custom AI development services, AI consulting, intelligent automation, and AI-powered business solutions.&lt;/P&gt;&lt;P&gt;I'm interested in understanding what businesses should prioritize when selecting an AI development company. How important are technical expertise, industry experience, scalability, data security, and ongoing support?&lt;/P&gt;&lt;P&gt;Has anyone worked with App Maisters or other AI development service providers? What was your experience, and what factors helped you determine whether investing in custom AI development was worthwhile?&lt;/P&gt;&lt;P&gt;I'd appreciate recommendations, real-world experiences, and insights from businesses that have successfully implemented AI solutions.&lt;/P&gt;</description>
      <pubDate>Wed, 11 Nov 2026 17:17:41 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/how-do-i-choose-the-right-ai-development-services-for-my/m-p/176197#M2268</guid>
      <dc:creator>jackoliver57</dc:creator>
      <dc:date>2026-11-11T17:17:41Z</dc:date>
    </item>
    <item>
      <title>Re: Seven checks before a Unity Gateway rollout</title>
      <link>https://community.databricks.com/t5/generative-ai/seven-checks-before-a-unity-gateway-rollout/m-p/176188#M2267</link>
      <description>&lt;P&gt;&lt;SPAN&gt;Thanks, &lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/125801"&gt;@DoTA&lt;/a&gt;&amp;nbsp;. A couple of distinctions I’d make here.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;If “blocked identity” means the caller lacks permissions, that tests access control, not a service-policy denial. For model services, the &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/data-governance/unity-catalog/service-policies/#policy-decisions" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;docs describe policy denials returning HTTP 200&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt; with &lt;/SPAN&gt;&lt;SPAN&gt;databricks_service_policy&lt;/SPAN&gt;&lt;SPAN&gt; details. So I wouldn’t assume the cases will both appear as 4xx. How that response is recorded in the usage table is still worth testing, as you suggest.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;Also, filtering on &lt;/SPAN&gt;&lt;SPAN&gt;tools/call&lt;/SPAN&gt;&lt;SPAN&gt; identifies tool-call requests, it doesn’t establish that the tool actually executed. An input policy could block it before execution. I’d keep attempted, blocked, and successful calls separate when defining the metrics.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;Have you observed how those blocked calls appear in the usage table? That would be useful to compare with the documented client behavior.&lt;/SPAN&gt;&lt;/P&gt;</description>
      <pubDate>Wed, 11 Nov 2026 15:15:09 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/seven-checks-before-a-unity-gateway-rollout/m-p/176188#M2267</guid>
      <dc:creator>Chalapathi-Rao</dc:creator>
      <dc:date>2026-11-11T15:15:09Z</dc:date>
    </item>
    <item>
      <title>Re: Seven checks before a Unity Gateway rollout</title>
      <link>https://community.databricks.com/t5/generative-ai/seven-checks-before-a-unity-gateway-rollout/m-p/176180#M2266</link>
      <description>&lt;P&gt;Good list. On the denials question, I couldn't find anywhere in the docs that says how a policy block shows up in the usage table compared with an upstream failure, so I'd test it before trusting any alert. Send one request from an identity that's blocked and one that hits a rate limit or a dead destination, then compare what lands in status_code and the mcp_metadata struct for each. If both come out as the same 4xx, have the client add its own request tag so you can split them afterwards.&lt;/P&gt;&lt;P&gt;For MCP, the usage rows carry json_rpc_method, so filter to rows where a tool is actually executed rather than the list and initialize traffic, otherwise your counts are mostly handshakes. Two things from the docs worth knowing early: the table is admin only by default (account and metastore admin), so whoever builds the alert probably needs a view or a grant first, and request tags sent in the header can't be backfilled, so settle the tag scheme before rollout, not after.&lt;/P&gt;&lt;P&gt;Also the usage tracking is itself billed, which is easy to miss on a checklist that has a cost item on it.&lt;/P&gt;</description>
      <pubDate>Wed, 11 Nov 2026 13:26:17 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/seven-checks-before-a-unity-gateway-rollout/m-p/176180#M2266</guid>
      <dc:creator>DoTA</dc:creator>
      <dc:date>2026-11-11T13:26:17Z</dc:date>
    </item>
    <item>
      <title>Dashboards dual axis</title>
      <link>https://community.databricks.com/t5/generative-ai/dashboards-dual-axis/m-p/176167#M2265</link>
      <description>&lt;P&gt;Hi!&lt;BR /&gt;&lt;BR /&gt;I am using AI/BI dashboards with dual y-axis. If I have Axis Range set to Default, it is not very good at auto-scaling.&amp;nbsp; See attached example - notice the huge amount of dead space&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="hhaaland_0-1794391222974.png" style="width: 400px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/33248iA21E897916019946/image-size/medium?v=v2&amp;amp;px=400" role="button" title="hhaaland_0-1794391222974.png" alt="hhaaland_0-1794391222974.png" /&gt;&lt;/span&gt;&lt;/P&gt;&lt;P&gt;I could of course chose custom range to get better result, but then if the data changes, then all or parts of the curves disappear since the range would be fixed.&lt;BR /&gt;&lt;BR /&gt;Do you know how to achieve better auto-scaling with dual-axis?&lt;/P&gt;</description>
      <pubDate>Wed, 11 Nov 2026 10:01:27 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/dashboards-dual-axis/m-p/176167#M2265</guid>
      <dc:creator>hhaaland</dc:creator>
      <dc:date>2026-11-11T10:01:27Z</dc:date>
    </item>
    <item>
      <title>Seven checks before a Unity Gateway rollout</title>
      <link>https://community.databricks.com/t5/generative-ai/seven-checks-before-a-unity-gateway-rollout/m-p/176104#M2264</link>
      <description>&lt;P&gt;&lt;SPAN&gt;Dear Community Members,&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;I’ve put together a practical guide to Databricks Unity Gateway, focusing on model access, MCP permissions, policy handling, and cost attribution.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;Here are seven checks I recommend before expanding a rollout:&lt;/SPAN&gt;&lt;/P&gt;&lt;OL&gt;&lt;LI&gt;&lt;P&gt;&lt;SPAN&gt;Verify that the intended identity can call the service and an unauthorized identity cannot.&lt;/SPAN&gt;&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;&lt;SPAN&gt;Check the service owner’s access to required destinations and credentials.&lt;/SPAN&gt;&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;&lt;SPAN&gt;Expose only the MCP tools the application needs.&lt;/SPAN&gt;&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;&lt;SPAN&gt;Make sure the client distinguishes policy denials from technical failures.&lt;/SPAN&gt;&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;&lt;SPAN&gt;Confirm that usage can be attributed to a caller, service, and workload.&lt;/SPAN&gt;&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;&lt;SPAN&gt;Test whether fallback responses still satisfy the application’s requirements.&lt;/SPAN&gt;&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;&lt;SPAN&gt;Define who receives budget alerts and what users see when usage is blocked.&lt;/SPAN&gt;&lt;/P&gt;&lt;/LI&gt;&lt;/OL&gt;&lt;P&gt;&lt;SPAN&gt;The walkthrough includes a runtime architecture diagram, a Python example, and links to official documentation.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;Full article:&amp;nbsp;&lt;A title="Databricks Unity Gateway—Access, Governance &amp;amp; Cost" href="https://medium.com/@chalapathi.rao.komirisetti/databricks-unity-gateway-who-called-your-ai-and-who-pays-fed5a4238291?sharedUserId=chalapathi.rao.komirisetti" target="_blank" rel="noopener"&gt;https://medium.com/@chalapathi.rao.komirisetti/databricks-unity-gateway-who-called-your-ai-and-who-pays-fed5a4238291?sharedUserId=chalapathi.rao.komirisetti&lt;/A&gt;&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;Which check has been the most challenging in your environment? I’d especially welcome examples of how you monitor policy denials and MCP permissions.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="runtime-architecture.png" style="width: 999px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/33226i044E132F800CA619/image-size/large?v=v2&amp;amp;px=999" role="button" title="runtime-architecture.png" alt="runtime-architecture.png" /&gt;&lt;/span&gt;&lt;/P&gt;</description>
      <pubDate>Tue, 10 Nov 2026 14:05:46 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/seven-checks-before-a-unity-gateway-rollout/m-p/176104#M2264</guid>
      <dc:creator>Chalapathi-Rao</dc:creator>
      <dc:date>2026-11-10T14:05:46Z</dc:date>
    </item>
    <item>
      <title>Re: How to size your Genie bill (Genie Agent, Genie Code) before the free period ends (31 January 20</title>
      <link>https://community.databricks.com/t5/generative-ai/how-to-size-your-genie-bill-genie-agent-genie-code-before-the/m-p/175910#M2263</link>
      <description>&lt;P&gt;The mid-December start date is the part most people will miss, so thanks for spelling it out.&lt;/P&gt;&lt;P&gt;Two things I noticed when I checked this against the Genie cost docs. The estimate query in step 4 prices the free rows by borrowing the most recent billed Genie SKU in the same workspace over the last 30 days. If your pilot workspace only has free usage (Genie One and Agents, nobody past the Genie Code allowance, no service principal), that lookup returns NULL and the cost column comes back empty. It is easy to read that as zero. Worth checking for it, or taking the price per DBU from a workspace that does have billed Genie rows.&lt;/P&gt;&lt;P&gt;I would also split the step 3 numbers by usage_metadata.genie.surface on top of the per-person split (identity_metadata.run_as has the user). Genie Code stays billed past the allowance while One and Agents are free until the end of January, so a team that mostly uses Code and a team that mostly uses Agents can show the same DBU total and behave very differently in February.&lt;/P&gt;&lt;P&gt;Also, the SQL block under step 3 did not render for me, there is just a blank. Might be worth re-editing the post in case others see the same.&lt;/P&gt;</description>
      <pubDate>Sun, 08 Nov 2026 12:52:58 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/how-to-size-your-genie-bill-genie-agent-genie-code-before-the/m-p/175910#M2263</guid>
      <dc:creator>DoTA</dc:creator>
      <dc:date>2026-11-08T12:52:58Z</dc:date>
    </item>
    <item>
      <title>Can you provide a list of Ai Chatbots that Specialize in handling government forms and applications?</title>
      <link>https://community.databricks.com/t5/generative-ai/can-you-provide-a-list-of-ai-chatbots-that-specialize-in/m-p/175830#M2262</link>
      <description>&lt;P&gt;Yes. Several AI-powered platforms are worth exploring if your goal is to simplify government forms, permit applications, and resident service requests.&lt;/P&gt;&lt;P&gt;Some options to consider include:&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;STRONG&gt;GovAssist&lt;/STRONG&gt; by &lt;STRONG&gt;App Maisters Government:&lt;/STRONG&gt; Supports government service workflows involving permits, licensing, benefits, taxes, and resident inquiries.&lt;/LI&gt;&lt;LI&gt;&lt;STRONG&gt;Vulcan Trajan:&lt;/STRONG&gt; Helps residents identify the correct government process, understand requirements, complete forms, and navigate permits and registrations.&lt;/LI&gt;&lt;LI&gt;&lt;STRONG&gt;TechForGov IGNA:&lt;/STRONG&gt; Combines AI assistance with digital forms, permit applications, notifications, and case management.&lt;/LI&gt;&lt;LI&gt;&lt;STRONG&gt;Ordinal:&lt;/STRONG&gt; Offers AI-powered assistance for local governments, helping residents find information through approved government documents and helping staff with permitting-related information.&lt;/LI&gt;&lt;/UL&gt;&lt;P&gt;When evaluating these solutions, prioritize features such as application guidance, document collection, eligibility checks, missing-information alerts, application status updates, and integration with existing government systems. These capabilities can help reduce incomplete submissions, repeated inquiries, and manual administrative work.&lt;/P&gt;</description>
      <pubDate>Fri, 06 Nov 2026 23:37:21 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/can-you-provide-a-list-of-ai-chatbots-that-specialize-in/m-p/175830#M2262</guid>
      <dc:creator>frankscharley3</dc:creator>
      <dc:date>2026-11-06T23:37:21Z</dc:date>
    </item>
    <item>
      <title>Re: Do non-ASCII characters in file names break document indexing over a Unity Catalog volume?</title>
      <link>https://community.databricks.com/t5/generative-ai/do-non-ascii-characters-in-file-names-break-document-indexing/m-p/175823#M2261</link>
      <description>&lt;P&gt;&lt;STRONG&gt;Update, as promised: it wasn't the file names.&lt;/STRONG&gt;&lt;/P&gt;&lt;P&gt;I had a support case open on this in parallel, and Databricks engineering has now looked into it. They reproduced å, ä, ö, spaces and accents in their own tests and everything retrieved correctly. They also found that my original volume, Swedish names and all, did answer correctly for a short while right after it was first indexed. So no renaming needed, and those of you who couldn't reproduce it were right.&lt;/P&gt;&lt;P&gt;What actually happened: about a day after the content search index was built, the serving table underneath it lost some required fields. From then on every search failed, and Genie shows that as "no relevant content" rather than an error. My ASCII copy only looked like the fix because it was a fresh index. That also explains the point raised here that all ~40 documents went silent, not just the six with Swedish names. Engineering is working on the root cause and a recovery.&lt;/P&gt;&lt;P&gt;Two things I learned along the way that might save someone else some time:&lt;/P&gt;&lt;OL&gt;&lt;LI&gt;Volume search in a Genie Agent only runs in Agent mode. My API calls that came back with "no documents are available in this space" were going through the non-agent path. If you call it programmatically, use the Agent mode APIs.&lt;/LI&gt;&lt;LI&gt;The index status kept saying every file was indexed with no errors while nothing could be retrieved. That's the part I'd most like to see fixed, so I've posted it as an idea: "Surface per-document extraction results in document indexing status, not just file counts". A kudo there helps it get noticed.&lt;/LI&gt;&lt;/OL&gt;&lt;P&gt;Since volume search is still in Public Preview, the suggestion I got for anyone on a tight timeline is the GA route: parse the PDFs with ai_parse_document, embed them with a multilingual model (Swedish matters for us), build an AI Search index over that table and expose it to the agent. It's more to build, but you get a retrieval layer you control and can check per document, which is exactly what was missing here.&lt;/P&gt;&lt;P&gt;Thanks again for all the replies and the testing. Knowing early that the names were unlikely to be the cause helped a lot.&lt;/P&gt;&lt;P&gt;Cheers,&lt;BR /&gt;Suranga&lt;/P&gt;</description>
      <pubDate>Fri, 06 Nov 2026 21:56:04 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/do-non-ascii-characters-in-file-names-break-document-indexing/m-p/175823#M2261</guid>
      <dc:creator>suranga</dc:creator>
      <dc:date>2026-11-06T21:56:04Z</dc:date>
    </item>
    <item>
      <title>Re: Do non-ASCII characters in file names break document indexing over a Unity Catalog volume?</title>
      <link>https://community.databricks.com/t5/generative-ai/do-non-ascii-characters-in-file-names-break-document-indexing/m-p/175781#M2260</link>
      <description>&lt;P&gt;Short answer from my test: the file names are not the problem. I ran ai_parse_document on a managed volume, and names with Swedish letters and spaces (both NFC and NFD forms) parsed fine, same as data_pulse found.&lt;/P&gt;&lt;P&gt;One thing worth knowing: parse failures don't fail the query. In my run the query succeeded for all 10 files, but 5 of them came back with zero elements and the reason only in error_status ("exceeds the limit of 500" pages, "Unsupported file format", "Document conversion failed"). A blank PDF returned nothing and no error. So "zero errors" may not mean what it looks like. This shows per-file counts:&lt;/P&gt;&lt;LI-CODE lang="python"&gt;SELECT element_at(split(path, '/'), -1) AS file,
size(ai_parse_document(content):document:elements::array&amp;lt;variant&amp;gt;) AS elements,
ai_parse_document(content):error_status::string AS error
FROM READ_FILES('/Volumes/&amp;lt;catalog&amp;gt;/&amp;lt;schema&amp;gt;/&amp;lt;volume&amp;gt;/', format =&amp;gt; 'binaryFile')
ORDER BY elements;&lt;/LI-CODE&gt;&lt;P&gt;If every file has elements and no error, parsing is fine and I'd look at indexing, sync or permissions next.&lt;/P&gt;</description>
      <pubDate>Fri, 06 Nov 2026 12:49:21 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/do-non-ascii-characters-in-file-names-break-document-indexing/m-p/175781#M2260</guid>
      <dc:creator>okinashchuk</dc:creator>
      <dc:date>2026-11-06T12:49:21Z</dc:date>
    </item>
    <item>
      <title>Re: 403: Permission Denied for certain foundational models</title>
      <link>https://community.databricks.com/t5/generative-ai/403-permission-denied-for-certain-foundational-models/m-p/175638#M2259</link>
      <description>&lt;P&gt;Hi&amp;nbsp;&lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/178839"&gt;@gautamrjt&lt;/a&gt;,&lt;/P&gt;
&lt;P&gt;I don't think this is a configuration issue on your side. Open models like Meta Llama 3.3 70B Instruct are available to pay-per-token workspaces by default. Premium partner models such as Anthropic Claude and the OpenAI GPT models have an extra account-level check, and Databricks has to verify your account before they're enabled. Your workspace being on the Premium tier doesn't unlock them because pricing tiers and account verification are handled separately. That's why one model works and the other returns 403 PERMISSION_DENIED, and why turning on Unity Gateway didn't help.&lt;BR /&gt;&lt;BR /&gt;&lt;/P&gt;
&lt;DIV class="du-bois-light-typography css-mdu5st" data-genai-markdown-block="true"&gt;Region isn't the cause either. The East U.S. region is a supported U.S. region for Claude Sonnet 5, so you don't need cross-Geo data processing. See&lt;SPAN&gt;&amp;nbsp;&lt;A href="https://learn.microsoft.com/en-us/azure/databricks/machine-learning/foundation-model-apis/compliance" target="_blank"&gt;this&lt;/A&gt;&lt;/SPAN&gt;&lt;SPAN&gt;&amp;nbsp;&lt;/SPAN&gt;for the regional availability table. On Azure, OpenAI models are provided through ADI Services, as described &lt;A href="https://learn.microsoft.com/en-us/azure/databricks/machine-learning/foundation-model-apis/priority-mode/" target="_blank"&gt;here&lt;/A&gt;.&lt;/DIV&gt;
&lt;DIV class="du-bois-light-typography css-mdu5st" data-genai-markdown-block="true"&gt;&amp;nbsp;&lt;/DIV&gt;
&lt;DIV class="du-bois-light-typography css-mdu5st" data-genai-markdown-block="true"&gt;&lt;SPAN&gt;Since this needs a change on the account side, you'll need to contact Databricks. Without a paid support plan you can still open a free Basic support request in the Azure portal under&amp;nbsp;&lt;/SPAN&gt;Help + support &amp;gt; Create a support request, choosing a subscription or account-related category. You can also email&amp;nbsp;&lt;A class="du-bois-light-typography css-1yqesw9" role="link" href="mailto:help@databricks.com" rel="noopener noreferrer" data-component-type="typography_link" data-component-id="codegen_webapp_js_genai_util_markdown.tsx_71" aria-disabled="false" target="_blank"&gt;help@databricks.com&lt;/A&gt;. Include your workspace ID, the full error, and the model names that fail and work, and ask for your account to be verified for partner-powered foundation models.&lt;/DIV&gt;
&lt;DIV class="du-bois-light-typography css-mdu5st" data-genai-markdown-block="true"&gt;&amp;nbsp;&lt;/DIV&gt;
&lt;DIV class="du-bois-light-typography css-mdu5st" data-genai-markdown-block="true"&gt;Before you raise the request, just rule out&lt;SPAN&gt;&amp;nbsp;two quick things. Fi&lt;/SPAN&gt;rst, make sure no AI Gateway rate limit is set on those endpoints. Second, check that your user has EXECUTE on the models insystem.ai. If both look fine, it's almost certainly the verification step.&lt;/DIV&gt;
&lt;DIV class="du-bois-light-typography css-mdu5st" data-genai-markdown-block="true"&gt;&amp;nbsp;&lt;/DIV&gt;
&lt;DIV class="du-bois-light-typography css-mdu5st" data-genai-markdown-block="true"&gt;
&lt;DIV class="du-bois-light-typography css-mdu5st" data-genai-markdown-block="true"&gt;You can keep building with the open models that already work, for example Llama 3.3 70B or GPT OSS 120B. Both are listed in&lt;SPAN&gt;&amp;nbsp;&lt;/SPAN&gt;&lt;A class="du-bois-light-typography css-1yqesw9" role="link" href="https://learn.microsoft.com/en-us/azure/databricks/machine-learning/foundation-model-apis/limits/" rel="noopener noreferrer" data-component-type="typography_link" data-component-id="codegen_webapp_js_genai_util_markdown.tsx_71" aria-disabled="false" target="_blank"&gt;Foundation Model APIs limits and quotas&lt;/A&gt;. If you need Claude or GPT right away, you can also connect them as external models using your own Anthropic or OpenAI API key and call them through the same OpenAI-compatible interface. The&lt;SPAN&gt;&amp;nbsp;&lt;/SPAN&gt;&lt;A class="du-bois-light-typography css-1yqesw9" role="link" href="https://learn.microsoft.com/en-us/azure/databricks/machine-learning/foundation-model-apis/index/" rel="noopener noreferrer" data-component-type="typography_link" data-component-id="codegen_webapp_js_genai_util_markdown.tsx_71" aria-disabled="false" target="_blank"&gt;Databricks Foundation Model APIs&lt;/A&gt;&lt;SPAN&gt;&amp;nbsp;&lt;/SPAN&gt;page has the query examples.&lt;/DIV&gt;
&lt;DIV class="du-bois-light-typography css-mdu5st" data-genai-markdown-block="true"&gt;&amp;nbsp;&lt;/DIV&gt;
&lt;DIV class="du-bois-light-typography css-mdu5st" data-genai-markdown-block="true"&gt;
&lt;P class="p1"&gt;&lt;FONT size="2" color="#FF6600"&gt;&lt;STRONG&gt;&lt;I&gt;If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.&lt;/I&gt;&lt;/STRONG&gt;&lt;/FONT&gt;&lt;I&gt;&lt;/I&gt;&lt;/P&gt;
&lt;/DIV&gt;
&lt;DIV class="du-bois-light-typography css-mdu5st" data-genai-markdown-block="true"&gt;Hope that gets you moving&lt;/DIV&gt;
&lt;/DIV&gt;</description>
      <pubDate>Wed, 04 Nov 2026 22:07:53 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/403-permission-denied-for-certain-foundational-models/m-p/175638#M2259</guid>
      <dc:creator>Ashwin_DSA</dc:creator>
      <dc:date>2026-11-04T22:07:53Z</dc:date>
    </item>
    <item>
      <title>403: Permission Denied for certain foundational models</title>
      <link>https://community.databricks.com/t5/generative-ai/403-permission-denied-for-certain-foundational-models/m-p/175632#M2258</link>
      <description>&lt;P&gt;Hello,&lt;/P&gt;&lt;P&gt;I have an individual account (Azure Databricks, Premier, East US)without any support contracts, and I am using it solely for personal projects. I am getting this error whenever I am trying to use the GPT and/or Claude models (system.ai.claude-sonnet-5-5). However, I am able to use models like "&lt;SPAN&gt;system.ai.meta-llama-3-3-70b-instruct&lt;/SPAN&gt;&lt;SPAN&gt;". I am getting this error for the models mentioned. I have enabled Unity Gateway. How do I overcome this?&lt;BR /&gt;&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;403 PERMISSION_DENIED&lt;/P&gt;&lt;P&gt;Access to this model is disabled for your workspace.&lt;BR /&gt;Your Databricks account may require verification&lt;BR /&gt;or additional model-access approval.&lt;/P&gt;&lt;P&gt;Contact Databricks Support or your Databricks&lt;BR /&gt;account team for assistance.&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;&amp;nbsp;Thanks.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;Gautam Radhakrishnan Ajit&lt;/SPAN&gt;&lt;/P&gt;</description>
      <pubDate>Wed, 04 Nov 2026 19:08:53 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/403-permission-denied-for-certain-foundational-models/m-p/175632#M2258</guid>
      <dc:creator>gautamrjt</dc:creator>
      <dc:date>2026-11-04T19:08:53Z</dc:date>
    </item>
    <item>
      <title>Re: Unable to Serve Gemini 3.8 Flash on Azure Databricks – East US</title>
      <link>https://community.databricks.com/t5/generative-ai/unable-to-serve-gemini-3-8-flash-on-azure-databricks-east-us/m-p/175622#M2257</link>
      <description>&lt;P data-pm-slice="1 3 []"&gt;Hi &lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/265091"&gt;@krSindhu&lt;/a&gt;, on Azure Databricks, &lt;A href="https://learn.microsoft.com/en-us/azure/databricks/machine-learning/foundation-model-apis/supported-models#gemini-3-8-flash" target="_blank"&gt;Gemini 3.8 Flash&lt;/A&gt; needs both ADI Services, which require eligibility and enablement, and cross-geography routing.&lt;/P&gt;
&lt;OL&gt;
&lt;LI&gt;
&lt;P&gt;Check that the &lt;A href="https://docs.databricks.com/adi" target="_blank"&gt;&lt;STRONG&gt;Integration with ADI Services&lt;/STRONG&gt;&lt;/A&gt; toggle is enabled.&lt;/P&gt;
&lt;/LI&gt;
&lt;LI&gt;
&lt;P&gt;Confirm that &lt;STRONG&gt;Enforce data processing within workspace Geography for Designated Services&lt;/STRONG&gt; is disabled on the workspace's &lt;STRONG&gt;Security and compliance&lt;/STRONG&gt; tab in the account console (&lt;A href="https://learn.microsoft.com/en-us/azure/databricks/resources/databricks-geos#enable-cross-geo-processing" target="_blank"&gt;enable cross-Geo processing&lt;/A&gt;).&lt;/P&gt;
&lt;/LI&gt;
&lt;LI&gt;
&lt;P&gt;Query &lt;CODE&gt;databricks-gemini-3-5-flash&lt;/CODE&gt;, an ADI Services model whose &lt;A href="https://learn.microsoft.com/en-us/azure/databricks/machine-learning/foundation-model-apis/supported-models#gemini-3-5-flash" target="_blank"&gt;model entry&lt;/A&gt; has no cross-geography note. If it responds, ADI Services is active and your account team can check 3.8 Flash access for your workspace. If it fails with the same errors as your 3.6 Flash screenshots, or the toggle isn't available, your account team can confirm ADI Services eligibility.&lt;/P&gt;
&lt;/LI&gt;
&lt;/OL&gt;</description>
      <pubDate>Wed, 04 Nov 2026 17:43:15 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/unable-to-serve-gemini-3-8-flash-on-azure-databricks-east-us/m-p/175622#M2257</guid>
      <dc:creator>AbhilashNagilla</dc:creator>
      <dc:date>2026-11-04T17:43:15Z</dc:date>
    </item>
    <item>
      <title>How to size your Genie bill (Genie Agent, Genie Code) before the free period ends (31 January 2027)</title>
      <link>https://community.databricks.com/t5/generative-ai/how-to-size-your-genie-bill-genie-agent-genie-code-before-the/m-p/175617#M2256</link>
      <description>&lt;P&gt;&lt;SPAN&gt;31 January 2027 is closer than it looks. Until then, every question your people ask Genie One or a Genie Agent is free. From 1 February, it's billed like the rest of Genie: a free monthly allowance per person, then pay-as-you-go.&lt;/SPAN&gt;&lt;/P&gt;
&lt;P&gt;&lt;SPAN&gt;If nobody has worked out the number by then, finance will find it on the first invoice. And the conversation won't be about rolling Genie out further. It will be about who switched it on.&lt;/SPAN&gt;&lt;/P&gt;
&lt;P&gt;&lt;SPAN&gt;There's also less time than the date suggests. &lt;STRONG&gt;A usage pilot ideally needs 30 days over a normal period (or a min of two weeks),&lt;/STRONG&gt;&amp;nbsp;with Christmas sitting in the middle, and you'll want a couple of weeks to agree limits with finance. Count back and &lt;/SPAN&gt;&lt;STRONG&gt;you need to start by mid-December&lt;/STRONG&gt;&lt;SPAN&gt;. Here's how to get a number you can defend.&lt;/SPAN&gt;&lt;/P&gt;
&lt;H3&gt;&lt;SPAN&gt;First, what's free and what isn't&lt;/SPAN&gt;&lt;/H3&gt;
&lt;UL&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;STRONG&gt;Free until 31 Jan:&lt;/STRONG&gt;&lt;SPAN&gt; Genie One and Genie Agents usage by people. During the promotion it doesn't use up the free allowance and doesn't count towards budgets.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;STRONG&gt;Free every month:&lt;/STRONG&gt;&lt;SPAN&gt; 150 DBUs per person, reset on the 1st.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;STRONG&gt;Already billed today:&lt;/STRONG&gt;
&lt;UL&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;SPAN&gt;Genie Code beyond the 150 DBUs, and &lt;/SPAN&gt;&lt;STRONG&gt;everything a service principal does&lt;/STRONG&gt;&lt;SPAN&gt;. Service principals get no free allowance and aren't covered by the promotion.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;SPAN&gt;The SQL warehouse that runs Genie's queries. It's billed separately, as before.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;/UL&gt;
&lt;/LI&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;SPAN&gt;&lt;STRONG&gt;Promotion:&lt;/STRONG&gt; 25% off billed Genie usage until 31 January 2027. The discount is built into the DBUs that are metered, not shown as a separate SKU.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;/UL&gt;
&lt;P&gt;&lt;SPAN&gt;All Genie usage is in system.billing.usage with billing_origin_product = 'GENIE'. Free usage is recorded under the GENIE_FREE_USAGE SKU (from 20 July 2026), and usage_metadata.genie.surface says whether it came from GENIE_CODE, GENIE_ONE or GENIE_AGENTS. That's what makes it possible to price it.&lt;/SPAN&gt;&lt;/P&gt;
&lt;H3&gt;&lt;SPAN&gt;Step-by-step&lt;/SPAN&gt;&lt;/H3&gt;
&lt;OL&gt;
&lt;LI&gt;&lt;STRONG&gt;STEP 0. Check you have admin access.&lt;/STRONG&gt; Budgets can only be created by an account admin, and the queries below need SELECT on system.billing.usage. Sort both out first, or you'll get stuck at step 1 or step 3.&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;STEP 1. Turn on an alert-only budget now.&lt;/STRONG&gt;&lt;SPAN&gt;&amp;nbsp;In the account console, go to &lt;/SPAN&gt;&lt;STRONG&gt;Usage → Budgets → Create budget&lt;/STRONG&gt;&lt;SPAN&gt;. Choose the &lt;/SPAN&gt;&lt;STRONG&gt;Unity Gateway&lt;/STRONG&gt;&lt;SPAN&gt; resource type and the &lt;/SPAN&gt;&lt;SPAN&gt;databricks-product: genie&lt;/SPAN&gt;&lt;SPAN&gt; tag, and don't add other tags. Add a shared threshold with only &lt;/SPAN&gt;&lt;STRONG&gt;Send alert&lt;/STRONG&gt;&lt;SPAN&gt; ticked, plus a lower one at about 80% as an early warning. Don't block anyone yet. Today this budget only tracks billed usage (Genie Code and service principals). Free usage never counts towards a budget, and a budget can't take away anyone's free allowance. It's the safety net you'll build on in step 6.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;STEP 2. Pick one representative team and run a one-month pilot (a calendar month).&lt;/STRONG&gt;&lt;SPAN&gt; Fourteen days is the minimum, and 30 is better. Choose a team that uses Genie for real work, not your most enthusiastic early adopters. Their usage is what you'll scale up from.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;STEP 3. Measure what the pilot would have cost.&lt;/STRONG&gt;&lt;SPAN&gt;&lt;SPAN&gt; Run this after the pilot month ends, or now to see last month. It adds up every Genie DBU per person, free and billed, and takes off the 150 DBU allowance once per person. It returns two rows (people and service principals) with DBUs per product, billable DBUs, a cost at your price per DBU, and the median and 75th percentile you'll use in step 5. It checks the last complete month by default; for a specific month, replace the two date lines.&lt;/SPAN&gt;&lt;/SPAN&gt;
&lt;OL&gt;
&lt;LI&gt;
&lt;P&gt;&lt;SPAN&gt;Keep the pilot inside one calendar month. The allowance resets on the 1st, so a pilot that spans two months needs this query run once per month.&amp;nbsp;&lt;/SPAN&gt;&lt;/P&gt;
&lt;/LI&gt;
&lt;LI&gt;The query&lt;/LI&gt;
&lt;LI&gt;&lt;LI-CODE lang="markup"&gt;&amp;lt;pre class="lia-code-sample language-sql"&amp;gt;
-- Genie usage over one pilot month, and what it would cost from February
WITH per_identity AS (
  SELECT identity_metadata.run_as LIKE '%@%'                                   AS is_person,
         identity_metadata.run_as                                              AS identity,
         SUM(IF(usage_metadata.genie.surface = 'GENIE_CODE',   usage_quantity, 0)) AS code_dbus,
         SUM(IF(usage_metadata.genie.surface = 'GENIE_ONE',    usage_quantity, 0)) AS one_dbus,
         SUM(IF(usage_metadata.genie.surface = 'GENIE_AGENTS', usage_quantity, 0)) AS agents_dbus,
         SUM(usage_quantity)                     AS dbus   -- free + billed, all record types
  FROM system.billing.usage
  WHERE billing_origin_product = 'GENIE'
       AND usage_date &amp;gt;= ADD_MONTHS(DATE_TRUNC('MONTH', CURRENT_DATE), -1)   -- last complete month
       AND usage_date &amp;lt;  DATE_TRUNC('MONTH', CURRENT_DATE)
  GROUP BY 1, 2)
SELECT IF(is_person, 'people', 'service principals')               AS who,
       COUNT(*)                                                    AS identities,
       ROUND(SUM(code_dbus))                                       AS code_dbus,
       ROUND(SUM(one_dbus))                                        AS one_dbus,
       ROUND(SUM(agents_dbus))                                     AS agents_dbus,
       ROUND(SUM(IF(is_person, GREATEST(dbus - 150, 0), dbus)))    AS billable_dbus,
       ROUND(SUM(IF(is_person, GREATEST(dbus - 150, 0), dbus)) * 0.07, 2) AS cost_usd,  -- your $ per DBU
       ROUND(percentile_approx(dbus, 0.5))                         AS median_dbus,
       ROUND(percentile_approx(dbus, 0.75))                        AS p75_dbus,
       ROUND(MAX(dbus))                                            AS heaviest_dbus
FROM per_identity
GROUP BY 1;
 &lt;/LI-CODE&gt;&lt;/LI&gt;
&lt;/OL&gt;
&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;STEP 4. Turn DBUs into a range, not a single number.&amp;nbsp;&lt;/STRONG&gt;&lt;SPAN&gt;The step 3 query already multiplies the billable DBUs by your price per DBU (0.07 in the example; replace it with your list or contract rate). Give finance two numbers: cost_usd × 0.75 (if the 25% promotion were extended) and cost_usd (list price). The docs don't say whether the discount continues after the promotion, so the range is the honest answer. One catch: Genie Code usage that's already billed today has the 25% built into its DBUs, so at full price it can cost up to a quarter more.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;STEP 5. Scale it up, then split people into three tiers.&lt;/STRONG&gt;&lt;SPAN&gt; Multiply by the number of people you expect to use Genie in February, not by everyone with access. Then group them the way the &lt;/SPAN&gt;&lt;A href="https://docs.databricks.com/aws/en/genie/consumption-guide" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;Genie consumption guide&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt; suggests:&lt;/SPAN&gt;
&lt;UL&gt;
&lt;LI&gt;&lt;STRONG&gt;Beginners:&lt;/STRONG&gt;&lt;SPAN&gt; occasional use. Most stay within the free allowance.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;Practitioners:&lt;/STRONG&gt;&lt;SPAN&gt; regular use. Set their limit from your median and 75th percentile.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;Power users:&lt;/STRONG&gt;&lt;SPAN&gt; long-running agent work. Highest limit, given as a group override.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI&gt;Create one account group per tier, so limits follow people rather than names on a spreadsheet.&lt;/LI&gt;
&lt;/UL&gt;
&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;STEP 6. Set per-person limits before 1 February.&lt;/STRONG&gt;&lt;SPAN&gt; Edit the budget from step 1. Keep the shared thresholds as alert-only, and add a &lt;/SPAN&gt;&lt;STRONG&gt;per-user threshold&lt;/STRONG&gt;&lt;SPAN&gt; for everyone (the beginner level), plus &lt;/SPAN&gt;&lt;STRONG&gt;group overrides&lt;/STRONG&gt;&lt;SPAN&gt; for practitioners and power users. Use &lt;/SPAN&gt;&lt;STRONG&gt;Block usage&lt;/STRONG&gt;&lt;SPAN&gt; on the per-user limits, not the shared one, so one heavy user can't block everyone.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI&gt;&lt;STRONG&gt;STEP 7. Check again in March.&lt;/STRONG&gt;&lt;SPAN&gt; Once February is complete, run the step 3 query and compare it with your estimate. Move people between tiers where the data says so.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;/OL&gt;
&lt;H3&gt;&lt;SPAN&gt;What good looks like (expected outcome)&lt;/SPAN&gt;&lt;/H3&gt;
&lt;UL&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;SPAN&gt;Finance has a monthly range for Genie before the promotion ends, with the query behind it.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;SPAN&gt;Every person sits in a tier, and each tier has a limit based on real usage, not guesses.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;SPAN&gt;One person's heavy month triggers an alert or a personal block, never a company-wide one.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;SPAN&gt;The first February invoice falls within the range you gave.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;/UL&gt;
&lt;H3&gt;&lt;SPAN&gt;Common errors (and how to avoid them)&lt;/SPAN&gt;&lt;/H3&gt;
&lt;UL&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;STRONG&gt;Reading the budget page as the bill.&lt;/STRONG&gt;&lt;SPAN&gt; During the promotion, Genie One and Genie Agents usage doesn't appear against budgets at all, and the page resets every month. Size from &lt;/SPAN&gt;&lt;SPAN&gt;system.billing.usage&lt;/SPAN&gt;&lt;SPAN&gt;.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;STRONG&gt;Forgetting service principals.&lt;/STRONG&gt;&lt;SPAN&gt; An app or integration running on one shared service principal is billed from day one, with no allowance, and every user's questions land on that one identity. Check for these first.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;STRONG&gt;Forgetting the SQL warehouse.&lt;/STRONG&gt;&lt;SPAN&gt; Genie budgets track only the LLM usage. The warehouse that runs Genie's SQL is billed separately, so size it too (I covered warehouse budgets &lt;/SPAN&gt;&lt;A href="https://community.databricks.com/t5/community-articles/how-to-set-budgets-and-alerts-on-sql-warehouses-so-self-service/td-p/172457" target="_blank" rel="noopener"&gt;&lt;SPAN&gt;here&lt;/SPAN&gt;&lt;/A&gt;&lt;SPAN&gt;).&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;STRONG&gt;Subtracting the allowance twice.&lt;/STRONG&gt;&lt;SPAN&gt; The query in step 3 subtracts it once from total usage. If you work from billed spend only, the allowance is already taken out.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;STRONG&gt;Setting beginner limits too low.&lt;/STRONG&gt;&lt;SPAN&gt; A limit that blocks someone in their first week stops them before they've found the value.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;LI style="font-weight: 400;" aria-level="1"&gt;&lt;STRONG&gt;Starting in January.&lt;/STRONG&gt;&lt;SPAN&gt; A two-week pilot over the new year won't show you a normal month.&lt;/SPAN&gt;&lt;/LI&gt;
&lt;/UL&gt;
&lt;H3&gt;&lt;SPAN&gt;The takeaway&lt;/SPAN&gt;&lt;/H3&gt;
&lt;P&gt;&lt;SPAN&gt;The free period isn't the time to stop measuring. It's the cheapest time you'll get to find out what Genie really costs, so finance hears a number from you before they see it on an invoice.&lt;/SPAN&gt;&lt;/P&gt;</description>
      <pubDate>Thu, 12 Nov 2026 12:35:42 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/how-to-size-your-genie-bill-genie-agent-genie-code-before-the/m-p/175617#M2256</guid>
      <dc:creator>Valeria_Koz_DBX</dc:creator>
      <dc:date>2026-11-12T12:35:42Z</dc:date>
    </item>
    <item>
      <title>Re: ai_decide() failing with Endpoint is not enabled for batch inference in Free Edition</title>
      <link>https://community.databricks.com/t5/generative-ai/ai-decide-failing-with-endpoint-is-not-enabled-for-batch/m-p/175361#M2255</link>
      <description>&lt;DIV&gt;&lt;DIV&gt;&lt;P&gt;The issue is tied to how Databricks Free Edition handles model serving resources. Specifically, the Free Edition imposes hard restrictions on these capabilities - there is no support for batch inference endpoints, no custom models&amp;nbsp;and certain foundation models are not available.&lt;/P&gt;&lt;P&gt;ai_decide() relies on an internal batch inference serving endpoint, Free Edition workspaces do not get that specific endpoint provisioned even if you have the partner powered AI features toggle enabled in the workspace settings. Notice the error in the stack trace (Endpoint ai_decide is not enabled for batch inference in this workspace)- it calls out the underlying serving endpoint rather than the preview feature. You are hitting the platform level provisioning restriction.&lt;/P&gt;&lt;/DIV&gt;&lt;/DIV&gt;</description>
      <pubDate>Mon, 02 Nov 2026 16:03:39 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/ai-decide-failing-with-endpoint-is-not-enabled-for-batch/m-p/175361#M2255</guid>
      <dc:creator>balajij8</dc:creator>
      <dc:date>2026-11-02T16:03:39Z</dc:date>
    </item>
    <item>
      <title>Re: Claude Desktop connect to Databricks MCP</title>
      <link>https://community.databricks.com/t5/generative-ai/claude-desktop-connect-to-databricks-mcp/m-p/175212#M2254</link>
      <description>&lt;P&gt;Hi&amp;nbsp;&lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/146197"&gt;@ShivaPolusani&lt;/a&gt;&amp;nbsp; - NOT sure whether you got your answer already or NOT....... but generally I followed the below steps for this activity -&amp;nbsp;&lt;BR /&gt;&lt;BR /&gt;&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;Path A: workspace with an account console (paid or trial)&lt;/STRONG&gt;&lt;/P&gt;&lt;OL&gt;&lt;LI&gt;In the Databricks &lt;STRONG&gt;account console&lt;/STRONG&gt;, go to Settings &amp;gt; App connections &amp;gt; Add connection.&lt;/LI&gt;&lt;LI&gt;Add Claude’s redirect URLs: &lt;A href="https://claude.ai/api/mcp/auth_callback" target="_blank" rel="noopener"&gt;https://claude.ai/api/mcp/auth_callback&lt;/A&gt; and &lt;A href="https://claude.com/api/mcp/auth_callback" target="_blank" rel="noopener"&gt;https://claude.com/api/mcp/auth_callback&lt;/A&gt;. Use a public client, and add the mcp.genie scope (or all-apis for testing). Save the &lt;STRONG&gt;Client ID&lt;/STRONG&gt;.&lt;/LI&gt;&lt;LI&gt;In Claude: Customizations &amp;gt; Connectors &amp;gt; Add custom connector. Use the URL https://&amp;lt;workspace-host&amp;gt;/api/2.0/mcp/genie/&amp;lt;genie-space-id&amp;gt;, open Advanced settings, and paste the Client ID.&lt;/LI&gt;&lt;LI&gt;Sign in when prompted. You need access to the Genie space and its SQL warehouse.&lt;/LI&gt;&lt;/OL&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;Path B: Databricks Free Edition (no account console, no personal access tokens)&lt;/STRONG&gt;&lt;/P&gt;&lt;OL&gt;&lt;LI&gt;Install Claude Desktop, Node.js (LTS) and the Databricks CLI.&lt;/LI&gt;&lt;LI&gt;Log in with the CLI: databricks auth login --host https://&amp;lt;workspace-host&amp;gt; --profile free_edition.&lt;/LI&gt;&lt;LI&gt;Create a launcher script, databricks-genie.cmd:&lt;/LI&gt;&lt;/OL&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&amp;nbsp;&lt;/DIV&gt;&lt;/DIV&gt;&lt;DIV&gt;bat&lt;/DIV&gt;&lt;DIV&gt;&lt;PRE&gt;&lt;SPAN&gt;   @echo off
   for /f "delims=" %%i in ('powershell -NoProfile -Command "(databricks auth token --profile free_edition | ConvertFrom-Json).access_token"') do set TOKEN=%%i
   set AUTH_HEADER=Bearer %TOKEN%
   npx mcp-remote https://&amp;lt;workspace-host&amp;gt;/api/2.0/mcp/genie/&amp;lt;genie-space-id&amp;gt; --header "Authorization:${AUTH_HEADER}"&lt;/SPAN&gt;&lt;/PRE&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;OL&gt;&lt;LI&gt;In claude_desktop_config.json, add:&lt;/LI&gt;&lt;/OL&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&amp;nbsp;&lt;/DIV&gt;&lt;/DIV&gt;&lt;DIV&gt;json&lt;/DIV&gt;&lt;DIV&gt;&lt;PRE&gt;&lt;SPAN&gt;   "databricks-genie": { "command": "C:\\path\\to\\databricks-genie.cmd", "args": [] }&lt;/SPAN&gt;&lt;/PRE&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;OL&gt;&lt;LI&gt;Fully quit Claude Desktop (system tray, Quit), reopen, start a new chat, and ask about your Genie space.&lt;/LI&gt;&lt;/OL&gt;&lt;P&gt;&lt;STRONG&gt;Gotchas we hit:&lt;/STRONG&gt;&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;The Genie space ID is only the part after /rooms/. Don’t include ?o=... from the browser URL.&lt;/LI&gt;&lt;LI&gt;Pasting a token from a web page can bring in an invisible zero-width character, which gives a “ByteString” error. Remove it or retype the token.&lt;/LI&gt;&lt;LI&gt;If PowerShell blocks npx, run Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned.&lt;/LI&gt;&lt;LI&gt;The Microsoft Store version of Claude Desktop keeps its config under AppData\Local\Packages\Claude_...\LocalCache\Roaming\Claude.&lt;/LI&gt;&lt;LI&gt;CLI tokens last about an hour. If the connector fails after a long session, restart Claude Desktop. If it still fails, run databricks auth login again.&lt;/LI&gt;&lt;LI&gt;This only exposes the one Genie space, not notebooks, jobs or dashboards.&lt;/LI&gt;&lt;/UL&gt;</description>
      <pubDate>Sat, 31 Oct 2026 17:02:47 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/claude-desktop-connect-to-databricks-mcp/m-p/175212#M2254</guid>
      <dc:creator>indradbxlearn</dc:creator>
      <dc:date>2026-10-31T17:02:47Z</dc:date>
    </item>
    <item>
      <title>Re: Do non-ASCII characters in file names break document indexing over a Unity Catalog volume?</title>
      <link>https://community.databricks.com/t5/generative-ai/do-non-ascii-characters-in-file-names-break-document-indexing/m-p/175211#M2253</link>
      <description>&lt;P&gt;&lt;SPAN&gt;Thanks for the write-up &lt;a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/195849"&gt;@suranga&lt;/a&gt;. I investigated this using managed UC volumes in Databricks&lt;/SPAN&gt;&lt;SPAN&gt;. I couldn't reproduce a non-ASCII filename issue, but the tests surfaced a few other ways documents can appear processed successfully while retrieval returns nothing.&lt;/SPAN&gt;&lt;/P&gt;&lt;H3&gt;&lt;SPAN&gt;1. Verify filenames and file content&lt;/SPAN&gt;&lt;/H3&gt;&lt;P&gt;&lt;SPAN&gt;I started with seven byte-identical PDFs using ASCII names, spaces, Swedish/Polish characters, NFC/NFD Unicode variants, and a literal %20.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;A useful first check:&lt;/SPAN&gt;&lt;/P&gt;&lt;PRE&gt;&lt;SPAN&gt;SELECT
    _metadata.file_name AS filename,
    length AS file_size,
    sha2(content, 256) AS content_hash
FROM read_files(
    '/Volumes/&amp;lt;catalog&amp;gt;/&amp;lt;schema&amp;gt;/&amp;lt;volume&amp;gt;/&amp;lt;dir&amp;gt;/',
    format =&amp;gt; 'binaryFile'
)
ORDER BY filename;&lt;/SPAN&gt;&lt;/PRE&gt;&lt;P&gt;&lt;STRONG&gt;&lt;SPAN&gt;Finding:&lt;/SPAN&gt;&lt;/STRONG&gt;&lt;SPAN&gt; All seven filenames were preserved, and the hashes matched. &lt;STRONG&gt;ai_parse_document&amp;nbsp;&lt;/STRONG&gt;&lt;/SPAN&gt;&lt;SPAN&gt;extracted identical text, &lt;STRONG&gt;ai_prep_search&amp;nbsp;&lt;/STRONG&gt;&lt;/SPAN&gt;&lt;SPAN&gt;generated nonempty chunks, and Delta Sync indexes successfully retrieved all seven records.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;This also worked when using the original Unicode filenames as primary keys with NFC and NFD remaining distinct.&lt;/SPAN&gt;&lt;/P&gt;&lt;H3&gt;&lt;SPAN&gt;2. Check extracted text and chunks, not just processing errors&lt;/SPAN&gt;&lt;/H3&gt;&lt;P&gt;&lt;SPAN&gt;I extended the test to 40 PDFs covering ASCII, Swedish characters, spaces, and NFC/NFD names. I also introduced malformed and blank documents to compare their behavior.&lt;/SPAN&gt;&lt;/P&gt;&lt;TABLE&gt;&lt;TBODY&gt;&lt;TR&gt;&lt;TD&gt;&lt;SPAN&gt;Scenario&lt;/SPAN&gt;&lt;/TD&gt;&lt;TD&gt;&lt;SPAN&gt;Nonempty extracted text&lt;/SPAN&gt;&lt;/TD&gt;&lt;TD&gt;&lt;SPAN&gt;Nonempty chunks&lt;/SPAN&gt;&lt;/TD&gt;&lt;TD&gt;&lt;SPAN&gt;observation&lt;/SPAN&gt;&lt;/TD&gt;&lt;/TR&gt;&lt;TR&gt;&lt;TD&gt;&lt;SPAN&gt;40 valid PDFs&lt;/SPAN&gt;&lt;/TD&gt;&lt;TD&gt;&lt;SPAN&gt;40/40&lt;/SPAN&gt;&lt;/TD&gt;&lt;TD&gt;&lt;SPAN&gt;40/40&lt;/SPAN&gt;&lt;/TD&gt;&lt;TD&gt;&lt;SPAN&gt;All passed&lt;/SPAN&gt;&lt;/TD&gt;&lt;/TR&gt;&lt;TR&gt;&lt;TD&gt;&lt;SPAN&gt;One malformed PDF substituted&lt;/SPAN&gt;&lt;/TD&gt;&lt;TD&gt;&lt;SPAN&gt;39/40&lt;/SPAN&gt;&lt;/TD&gt;&lt;TD&gt;&lt;SPAN&gt;39/40&lt;/SPAN&gt;&lt;/TD&gt;&lt;TD&gt;&lt;SPAN&gt;Per-file conversion error&lt;/SPAN&gt;&lt;/TD&gt;&lt;/TR&gt;&lt;TR&gt;&lt;TD&gt;&lt;SPAN&gt;Valid blank PDF&lt;/SPAN&gt;&lt;/TD&gt;&lt;TD&gt;&lt;SPAN&gt;0/1&lt;/SPAN&gt;&lt;/TD&gt;&lt;TD&gt;&lt;SPAN&gt;0/1&lt;/SPAN&gt;&lt;/TD&gt;&lt;TD&gt;&lt;SPAN&gt;No parse/prep error&lt;/SPAN&gt;&lt;/TD&gt;&lt;/TR&gt;&lt;/TBODY&gt;&lt;/TABLE&gt;&lt;P&gt;&lt;STRONG&gt;&lt;SPAN&gt;The blank PDF was the interesting case:&lt;/SPAN&gt;&lt;/STRONG&gt;&lt;SPAN&gt; parsing and preparation reported no errors, but produced zero text and zero retrieval chunks.A processed-files count alone doesn’t confirm that usable content reached the indexing stage, so I’d inspect output per file:&lt;/SPAN&gt;&lt;/P&gt;&lt;PRE&gt;&lt;SPAN&gt;WITH parsed AS (
    SELECT
        _metadata.file_name AS filename,
        ai_parse_document(content) AS parsed_doc
    FROM read_files(
        '/Volumes/&amp;lt;catalog&amp;gt;/&amp;lt;schema&amp;gt;/&amp;lt;volume&amp;gt;/&amp;lt;dir&amp;gt;/',
        format =&amp;gt; 'binaryFile'
    )
)
SELECT
    filename,
    parsed_doc:error_status AS parse_error,
    ai_prep_search(parsed_doc) AS prepared
FROM parsed;&lt;/SPAN&gt;&lt;/PRE&gt;&lt;P&gt;Check the extracted content,&amp;nbsp;prepared.error_status, and nonempty chunk_to_retrieve values.&lt;/P&gt;&lt;H3&gt;&lt;SPAN&gt;3. Query the index directly before troubleshooting the assistant&lt;/SPAN&gt;&lt;/H3&gt;&lt;P&gt;&lt;SPAN&gt;I tested direct VECTOR and HYBRID queries against Delta Sync indexes, then compared their results with downstream filtering.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;&lt;SPAN&gt;Findings:&lt;/SPAN&gt;&lt;/STRONG&gt;&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;SPAN&gt;Direct VECTOR and HYBRID retrieval: PASS.&lt;/SPAN&gt;&lt;/LI&gt;&lt;LI&gt;&lt;SPAN&gt;Correct downstream filter: 7 records retained.&lt;/SPAN&gt;&lt;/LI&gt;&lt;LI&gt;&lt;SPAN&gt;Deliberately incorrect local filter: 0 records retained.&lt;/SPAN&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;P&gt;&lt;SPAN&gt;The filter case was a controlled simulation, but it demonstrates how valid index results can disappear downstream.&lt;/SPAN&gt;&lt;/P&gt;&lt;P&gt;&lt;SPAN&gt;For a Vector Search index with Databricks-managed query embeddings, you can test directly:&lt;/SPAN&gt;&lt;/P&gt;&lt;PRE&gt;&lt;SPAN&gt;from databricks.vector_search.client import VectorSearchClient

index = VectorSearchClient().get_index(
    endpoint_name="&amp;lt;endpoint&amp;gt;",
    index_name="&amp;lt;catalog&amp;gt;.&amp;lt;schema&amp;gt;.&amp;lt;index&amp;gt;"
)

result = index.similarity_search(
    query_text="&amp;lt;distinctive phrase from PDF&amp;gt;",
    columns=["&amp;lt;id_column&amp;gt;", "&amp;lt;text_column&amp;gt;"],
    num_results=10,
    query_type="HYBRID"
)
print(result)&lt;/SPAN&gt;&lt;/PRE&gt;&lt;P&gt;&lt;SPAN&gt;For self-managed embeddings, provide a compatible &lt;STRONG&gt;query_vector&amp;nbsp;&lt;/STRONG&gt;&lt;/SPAN&gt;&lt;SPAN&gt;as well.&amp;nbsp;&lt;/SPAN&gt;&lt;SPAN&gt;If direct search works but the assistant returns nothing, inspect source selection, metadata filters, document IDs, and path comparisons.&lt;/SPAN&gt;&lt;/P&gt;&lt;H3&gt;&lt;SPAN&gt;4. Isolate files and verify synchronization&lt;/SPAN&gt;&lt;/H3&gt;&lt;P&gt;&lt;SPAN&gt;Two additional checks were useful:&lt;/SPAN&gt;&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;STRONG&gt;Batch isolation:&lt;/STRONG&gt; split the collection into smaller batches and check per-file extraction and chunk counts. In a controlled fault-injection test, recursive bisection isolated a deliberately failing file in four steps.&lt;/LI&gt;&lt;LI&gt;&lt;STRONG&gt;Index sync:&lt;/STRONG&gt; initial and incremental Delta Sync updates both reached &lt;STRONG&gt;COMPLETED&lt;/STRONG&gt;, and a newly appended marker record became retrievable. So check actual sync completion and record visibility, not just file-processing status.&lt;/LI&gt;&lt;LI&gt;&lt;STRONG&gt;Permissions:&lt;/STRONG&gt; verify the relevant ingestion and query identities, starting with the command below. Include inherited catalog and schema privileges and the search index’s own access permissions where applicable. I only verified my own identity.&lt;/LI&gt;&lt;/UL&gt;&lt;PRE&gt;&lt;SPAN&gt;SHOW GRANTS ON VOLUME &amp;lt;catalog&amp;gt;.&amp;lt;schema&amp;gt;.&amp;lt;volume&amp;gt;;&lt;/SPAN&gt;&lt;/PRE&gt;&lt;H3&gt;Your three questions&lt;/H3&gt;&lt;P&gt;&lt;STRONG&gt;1. Non-ASCII names breaking indexing&lt;/STRONG&gt;&lt;BR /&gt;Not in my tests: Swedish and Polish characters, spaces, NFC/NFD forms and a literal %20 all survived parsing, chunking and Delta Sync indexing. That covers small PDFs and the building-block functions only, so I can’t rule it out for Knowledge Assistant or Genie volume search. I also haven’t found it documented as a known limit.&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;2. Normalising to ASCII?&lt;/STRONG&gt;&lt;BR /&gt;I wouldn’t call it required based on what I saw, so I wouldn’t rename your originals as a first step. What I’d do is stop depending on the name:&lt;STRONG&gt; generate an ID at ingestion and let nothing downstream use the filename as a key, path or filter value.&lt;/STRONG&gt; That avoids the NFC/NFD and encoding mismatches that can appear wherever a name is converted into a URL, key or filter. If you do transform names, don’t normalise the string while the file on disk keeps its old name, or the new string no longer points at the stored object.&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;3. Where do you keep the readable original title?&lt;/STRONG&gt;&lt;BR /&gt;In a column next to the parsed text, keyed by a generated ID:&lt;/P&gt;&lt;LI-CODE lang="markup"&gt;SELECT
    sha2(content, 256)  AS doc_id,         -- stable ASCII key
    _metadata.file_name AS original_name   -- readable title for display
FROM read_files('/Volumes/catalog/schema/volume/dir/', format =&amp;gt; 'binaryFile')&lt;/LI-CODE&gt;&lt;P&gt;Index on &lt;STRONG&gt;doc_id&lt;/STRONG&gt; and show &lt;STRONG&gt;original_name&amp;nbsp;&lt;/STRONG&gt;to users. In my test this worked, and so did raw filenames as primary keys, so it’s good hygiene rather than a fix. A content hash gives duplicate PDFs the same ID, so add the path to the hash if you have copies under different names.&lt;/P&gt;&lt;H3&gt;What stands out in your case&lt;/H3&gt;&lt;P&gt;Only &lt;STRONG&gt;6 of ~40 PDFs had Swedish characters&lt;/STRONG&gt;, yet retrieval was empty for the entire collection. If the names were the cause, the 34 ASCII-named files should still have worked, so I’m particularly interested in a collection-wide ingestion, synchronization or retrieval configuration issue.&lt;/P&gt;&lt;P&gt;I’d troubleshoot in this order:&lt;/P&gt;&lt;OL&gt;&lt;LI&gt;Check file discovery and extracted text and chunk counts per PDF. If some fail, isolate them in smaller batches.&lt;/LI&gt;&lt;LI&gt;Check index synchronization and whether the expected records are present.&lt;/LI&gt;&lt;LI&gt;Query a distinctive phrase from an ASCII-named PDF directly against the index.&lt;/LI&gt;&lt;LI&gt;If direct retrieval works, inspect assistant-side source selection and filters.&lt;/LI&gt;&lt;/OL&gt;&lt;P&gt;&lt;STRONG&gt;Limits of these tests&lt;/STRONG&gt;&lt;BR /&gt;These came from small controlled synthetic PDFs and Delta Sync indexes, not your 77,000-character documents, so size-related limits are untested. They don’t establish the cause in your managed indexing workflow and I couldn’t exercise the Knowledge Assistant or Genie managed volume path in my environment.&lt;/P&gt;</description>
      <pubDate>Sat, 31 Oct 2026 16:40:06 GMT</pubDate>
      <guid>https://community.databricks.com/t5/generative-ai/do-non-ascii-characters-in-file-names-break-document-indexing/m-p/175211#M2253</guid>
      <dc:creator>data_pulse</dc:creator>
      <dc:date>2026-10-31T16:40:06Z</dc:date>
    </item>
  </channel>
</rss>

