All Generative AI posts

Re: Genie Agent SDK visualization

Louis_Frolio — Thu, 01 Oct 2026 19:17:34 GMT

Hello @haaland, I took a look at both internal and external documentation and here is what I found.

Glad it's working, and @Gecofer and @ivanvyd had the right diagnosis. enable_visualization was never removed. It shipped in databricks-sdk v0.120.0 (July 2026), alongside the viz attachment type and download_message_attachment_visualization(), and nothing through 0.141.0 took it back out. So a 0.141.0 version string next to a signature that lacks the argument means the files on disk and the code running in your Python process had drifted apart: either a stale process still holding the classes your WorkspaceClient was built from, or a second interpreter or SDK copy doing the loading.

The checks Gecofer and Ivan suggested are the way to confirm which:

import inspect, sys
import databricks.sdk

print(sys.executable)
print(databricks.sdk.__version__)
print(databricks.sdk.__file__)
print(inspect.signature(w.genie.start_conversation_and_wait))

In a notebook, upgrade, restart Python, then rebuild the client:

%pip install --upgrade databricks-sdk
dbutils.library.restartPython()

After the restart the signature should show enable_visualization: Optional[bool] = None. If it still doesn't, compare sys.executable and databricks.sdk.__file__ against the environment where you ran the install. For a Databricks App, pin databricks-sdk>=0.120.0 in requirements.txt and redeploy.

On what the flag actually does: it asks Genie to generate a visualization but doesn't guarantee one. When the question and data support a chart, the response carries a viz attachment (Beta) next to the query attachment, with a title and the query_attachment_id it came from. For a rendered PNG, call w.genie.download_message_attachment_visualization(name=...) with name in the form spaces/{space_id}/conversations/{conversation_id}/messages/{message_id}/attachments/{attachment_id}. That only works once the message status is COMPLETED, so the _and_wait variants you're using are the right fit.

References:

Genie SDK reference: https://databricks-sdk-py.readthedocs.io/en/stable/workspace/dashboards/genie.html
SDK release notes (v0.120.0 added enable_visualization😞 https://github.com/databricks/databricks-sdk-py/releases
Genie Conversation API reference: https://docs.databricks.com/api/genie/v1/conversation
Databricks SDK for Python on PyPI: https://pypi.org/project/databricks-sdk/

Regards,
Louis.

Re: Gemini Foundation Model endpoint unavailable in 14-day commercial trial

Louis_Frolio — Thu, 01 Oct 2026 19:06:50 GMT

Hello @Remon , I took a look at both internal and external documentation and here is what I found.

This looks like a model availability or routing issue, not a missing custom endpoint. Seeing databricks-gemini-3-8-flash under system.ai doesn't mean it's enabled for your workspace. That catalog is a global list of everything Databricks hosts anywhere. Whether a callable model service exists in a given workspace depends on region, cross-Geo settings, serving capacity, and your permissions. The docs say this plainly on the Unity Gateway models page, so NotFound here is consistent with "listed but not materialized," not a broken workspace.

The most likely cause is cross-geography routing. Gemini 3.8 Flash is hosted on a global endpoint, and its model page states it requires cross geography routing to be enabled. Your Tokyo workspace is in the Asia Geo, so requests have to be allowed to leave that Geo. It's normally on by default outside the US and EU, but it's off if a compliance security profile is enabled, and it's a per-workspace toggle that's easy to miss. To check:

Sign in to the account console (https://accounts.cloud.databricks.com/) as an account admin. If you set up the trial, that should be you.
Click Workspaces and open your Tokyo workspace.
Open the Security and compliance tab.
Make sure "Enforce data processing within workspace Geography for Designated Services" is turned off.

Then look for the system.ai.gemini-3-8-flash model service in Unity Gateway and confirm your user can query it.

Second, check the request shape. There are three API paths now, and mixing identifiers across them produces exactly the error you quoted:

Unity Gateway unified API: POST to https://<workspace-url>/ai-gateway/mlflow/v1/chat/completions with "model": "system.ai.gemini-3-8-flash"
Native Gemini API: https://<workspace-url>/ai-gateway/gemini/v1beta/models/system.ai.gemini-3-8-flash:generateContent
Legacy Model Serving: https://<workspace-url>/serving-endpoints/databricks-gemini-3-8-flash/invocations

Sending system.ai.gemini-3-8-flash to /serving-endpoints/, or databricks-gemini-3-8-flash to /ai-gateway/, fails with NotFound even when everything is provisioned correctly. Pay-per-token models are managed services, so you shouldn't need to create any endpoint yourself.

A quick isolation test: try system.ai.gemini-3-7-flash or system.ai.claude-sonnet-4-6 first. Both are listed for ap-northeast-1 without the cross-Geo requirement. If those work and 3.8 Flash doesn't, it's the Geo setting. If none work, it's the request shape or entitlements.

Your four questions, in order:

Trial availability: neither the public docs nor our internal references confirm or deny a trial-specific restriction on this model. If region, routing, identifier, and permissions all check out and it's still absent, that's the point where Databricks would need to verify trial entitlement or backend provisioning.
Listed but not callable: global catalog listing versus regional availability, as above.
Entitlements or provisioning: you need Unity Catalog enabled, the Workspace access entitlement on your user, and the cross-Geo setting. No ticket should be needed.
Workspace setting: the account console toggle above. A metastore admin can also restrict which system.ai models users may query, though on a fresh trial that's unlikely to have changed.

One caveat. The region availability table lists databricks-gemini-3-8-flash for ap-northeast-1 without the cross-Geo marker, while the model's own page says it requires it. The two pages aren't perfectly consistent, so the toggle is the practical test. If you flip it, fix the request shape, and still get NotFound, reply here with your cloud and region, the exact identifier and API path, a timestamp, and the sanitized error (no tokens or credentials, please). I'll take another look.

References:

Available models in Unity Gateway (note on system.ai listing): https://docs.databricks.com/aws/en/machine-learning/model-serving/foundation-model-overview
Gemini 3.8 Flash model page: https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/supported-models#gemini-3-8-flash
Model availability by region: https://docs.databricks.com/aws/en/unity-gateway/model-region-availability
Databricks Geos and cross-Geo processing: https://docs.databricks.com/aws/en/resources/databricks-geos#cross-geo-processing
Query model services through Unity Gateway: https://docs.databricks.com/aws/en/ai-gateway/query-model-services
Query with the Google Gemini API: https://docs.databricks.com/aws/en/machine-learning/model-serving/query-gemini-api
Use foundation models (legacy serving-endpoint query options): https://docs.databricks.com/aws/en/machine-learning/model-serving/score-foundation-models

Regards,
Louis.

Re: WORKSPACE HAS NO DAILY TOKEN ALLOWANCE

HOguns — Thu, 01 Oct 2026 16:08:42 GMT

Hello @ErikVetter.

Thank you so much; that worked like magic.

Regards

Gemini Foundation Model endpoint unavailable in 14-day commercial trial

Remon — Thu, 01 Oct 2026 13:48:08 GMT

Hi,

I am using a 14-day Databricks commercial trial with promotional credits on an AWS Tokyo workspace

I am specifically trying to access the documented "databricks-gemini-3-8-flash" Foundation Model, but it is not available through Unity Gateway / Model Serving in my workspace.

What I have already checked:

- "databricks-gemini-3-8-flash" appears under the "system.ai" catalog / registered models.

- My workspace is in the AWS Tokyo region.

- I checked the relevant workspace AI and feature settings.

- I checked Unity Gateway / Model Serving, but I cannot find a Gemini model service or callable endpoint.

- I also tried calling the model endpoint directly through the API.

The API returns:

"NotFound: The given endpoint does not exist, please retry after checking the specified model and version deployment exists."

I contacted Databricks Support, but they informed me that my account is classified as an individual account without a support contract and recommended that I post the issue here in the Community.

Could someone from Databricks please clarify:

1. Is "databricks-gemini-3-8-flash" available on the 14-day commercial trial?

2. If it is available, why does the model appear in "system.ai" but not as a callable endpoint in my workspace?

3. Does my workspace require any additional entitlement, backend provisioning, or activation?

4. Is there any workspace setting I need to enable, or does Databricks need to provision the model for my account?

Thank you.

Routing between GLM 5.3 Flash and GLM 5.3 with ai_decide: a cheap router in front of ai_query

DoTA — Thu, 01 Oct 2026 13:21:43 GMT

I hope this helps for those who are trying to keep LLM costs under control on batch workloads. I have been looking at ai_decide (Beta) as a router rather than as a classifier, and it fits that job well. Sharing the pattern and the open questions, since I have not benchmarked it at scale yet.

THE IDEA

Most rows in a typical batch job (ticket triage, review summaries, document extraction) do not need the big model. A small, fast decision step can look at each row and choose which model should handle it. ai_decide takes text plus a set of questions and returns a choice, a probability or a score, and it is documented as faster and cheaper than a general LLM for decisions. The Databricks announcement lists "routing prompts to the right model" as a use case.

Endpoints (from the supported models page; re-check in your workspace with WorkspaceClient().serving_endpoints.list(), because names change quickly):

- databricks-glm-5-3-flash : cheaper, multimodal, reasoning always on

- databricks-glm-5-3 : the full model, text only, reasoning effort configurable

STEP 1: ask ai_decide for a tier and a confidence

ai_decide(state, questions, options) returns a VARIANT. For a choice question you get the chosen label, per-label probabilities and a confidence.

SELECT id, prompt,

ai_decide(

prompt,

"tier": {

"type": "choice",

"instructions": "Which model tier is needed to answer this request well?",

"criteria": {

"flash": "Short factual lookup, simple extraction, classification, or rewriting. No multi-step reasoning.",

"full": "Multi-step reasoning, ambiguous or conflicting inputs, long synthesis, code generation, or anything where an error is costly."

}

}',

map('version', '1.0')

) AS d

FROM prompts_to_process;

Pull the fields out of the VARIANT:

d:response:answers:tier:choice::string -- 'flash' or 'full'

d:response:answers:tier:confidence::double -- how sure the router is

STEP 2: send each row to the model it was routed to

ai_query needs the endpoint name as a constant, so you cannot pass the routed name in as a column value. Two options: (a) a CASE with one ai_query call per branch, or (b) two statements, one filtered on tier = 'flash' and one on tier = 'full', writing into the same results table. I prefer (b): each batch is homogeneous, you can size and monitor each endpoint separately, and you do not depend on how the engine evaluates both branches of a CASE.

CREATE OR REPLACE TABLE routed AS

SELECT id, prompt,

d:response:answers:tier:choice::string AS tier,

d:response:answers:tier:confidence::double AS conf

FROM (SELECT id, prompt, ai_decide(prompt, '<questions json from step 1>', map('version','1.0')) AS d

FROM prompts_to_process);

-- cheap path

INSERT INTO results

SELECT id, 'flash' AS model, ai_query('databricks-glm-5-3-flash', prompt) AS answer

FROM routed WHERE tier = 'flash' AND conf >= 0.7;

-- everything else goes to the full model, including low-confidence rows

INSERT INTO results

SELECT id, 'full' AS model, ai_query('databricks-glm-5-3', prompt) AS answer

FROM routed WHERE tier = 'full' OR conf < 0.7;

The design choice I think matters most: send low-confidence rows to the full model. A router that is unsure should fail expensive, not fail wrong. Tune the 0.7 threshold on your own data.

THINGS TO CHECK BEFORE TRUSTING THIS IN PRODUCTION

1. Label a sample (a few hundred rows) by running both models and comparing. The router only saves money if flash answers are good enough on the rows it keeps. Measure answer quality per tier, not just router accuracy.

2. Watch the cost of the router itself. ai_decide is cheap per call, but on very short prompts it may be a meaningful share of the flash call.

3. Log tier, confidence and model per row so you can see the real flash/full split and recompute savings.

4. ai_decide is Beta, only available in some regions, and not on Databricks SQL Classic, so check your warehouse type and region first.

5. Re-verify endpoint names and pricing. Both are pay-per-token endpoints and the lineup moves fast.

WHAT I HAVE NOT TESTED

I have not benchmarked routing accuracy or the real cost split at scale, so treat the threshold and the criteria wording as a starting point. The criteria text moves results the most, so it is worth iterating on it with a labelled sample.

QUESTIONS FOR THE COMMUNITY

- Has anyone tried ai_decide as a router at volume? What flash/full split did you land on?

- Any experience with score-type questions (a 0 to 2 difficulty score) versus a choice question for routing? I picked choice because it also returns a confidence I can threshold on.

- Are you routing on prompt content alone, or also on metadata such as customer tier or SLA?

References: ai_decide SQL function reference and announcement blog, and the Foundation Model APIs supported models page on docs.databricks.com.

Re: WORKSPACE HAS NO DAILY TOKEN ALLOWANCE

ErikVetter — Thu, 01 Oct 2026 08:41:34 GMT

Hey, I had a simillar problem, with the same Error Code.
I asked Codex to fix it and the solution is very simple.

In the Genie Code window, next to the send button, change from auto to low.
I dont know, this seems like a bug since some weeks to me.

Re: WORKSPACE HAS NO DAILY TOKEN ALLOWANCE

HOguns — Wed, 30 Sep 2026 17:39:27 GMT

Alright, thank you. I will have to delete the workspace now because its running on a pay as you go and I am still stuck.

Re: WORKSPACE HAS NO DAILY TOKEN ALLOWANCE

Valeria_Koz_DBX — Wed, 30 Sep 2026 12:17:36 GMT

Hi @HOguns,

I am not an engineer, I cover the delivery side of Genie.

However, I passed it to one of our internal teams. Let's see if they pick it up. I am not sure how engineers pick it up from here

In the meantime, can you also please send it to databricks-community@databricks.com, so it is not missed out on

Re: WORKSPACE HAS NO DAILY TOKEN ALLOWANCE

HOguns — Tue, 29 Sep 2026 18:01:56 GMT

Hello @Valeria_Koz_DBX

I contacted the Help Center via email since I am unable to do point 1 above to open a case. The response from your team via email was to post it here, and one of the engineers would help instead.

Regards

Re: WORKSPACE HAS NO DAILY TOKEN ALLOWANCE

Valeria_Koz_DBX — Tue, 29 Sep 2026 16:44:25 GMT

@HOguns

Did you posted here, or Contacted Help Center with all of the details above?

Ideally, you want to follow the below steps

Use the Databricks Help Center if you can access it, and open a case for Azure Databricks → Genie Code/Assistant.
If Help Center access is denied, email help@databricks.com from the same address used for the Databricks account. Help Center access may require an authorized support contact or support entitlement.

Re: WORKSPACE HAS NO DAILY TOKEN ALLOWANCE

HOguns — Tue, 29 Sep 2026 15:53:35 GMT

Hi @Valeria_Koz_DBX,

Thank you for your prompt feedback; I have done exactly as you highlighted in No. 1 with the service principal name "mondayhenry31_gmail.com#EXT#@mondayhenry31gmail.onmicrosoft.com" and I have used this to sign in using the in-Private/incognito mode, but I still hit the same impasse. Based on No 2.

Databricks account ID - 06348851-c57a-4116-9779-6baae571c2a1

Workspace ID and workspace URL - 7405617987563324

Azure region - East US

The exact identity/UPN used to sign in - mondayhenry31_gmail.com#EXT#@mondayhenry31gmail.onmicrosoft.com

Approximate time of the failure and a screenshot of the error - 11:40 Canada Time EDT

Confirmation that the user identity and workspace assignment were checked - Yes it was checked

Below I have provided the workspace details as well

Name

Status

Resource group

Region

Subscription

Created

Metastore

adb-dp750

Running

rg-dp750

eastus

06348851-c57a-4116-9779-6baae571c2a1

yesterday at 3:37 PM

metastore_azure_eastusOpen

Regard

Re: WORKSPACE HAS NO DAILY TOKEN ALLOWANCE

Valeria_Koz_DBX — Tue, 29 Sep 2026 15:04:31 GMT

Hello @HOguns ,

Long response, but it is worth it.

Thank you for sharing the screenshots. There appear to be two separate issues: the Genie Code daily-token allowance and Help Center access.

1. Confirm the Databricks identity being used

The screenshots show several external identities for the same person, including mondayhenry31@gmail.com, an Azure guest identity containing #ext#, and onmicrosoft.com identities. The Account admin role appears to be assigned to the #ext# identity, not clearly to the plain Gmail identity.

Please ask your Microsoft Entra ID administrator to confirm which user principal name (UPN) is used when you sign in to Azure Databricks. Then:

Sign out of Databricks and Microsoft accounts in the browser, or use an incognito window.
Sign in using the same identity that has the Databricks Account admin role.
Confirm that this identity is assigned to the affected workspace.
If workspace administration is required, confirm that it also has the Workspace admin role.

Account-admin access and workspace access are separate permissions. Azure Databricks also recommends avoiding duplicate identities and using one identity-provisioning method. Do not delete the duplicate entries until the canonical Entra ID identity has been confirmed.

References: Azure Databricks account administration, Manage users in Azure Databricks, and Automatic identity management.

2. The budget does not create the daily token allowance

The screenshot confirms that Genie Code is displaying the error:

This workspace has no daily token allowance for the assistant. Ask your workspace admin to enable it, then try again.

A Genie budget is a usage-cost control; creating a budget does not by itself provision or enable the assistant’s daily token allowance. If a budget is configured, verify that it is scoped to the Genie product using the documented Unity Gateway resource and databricks-product = genie tag.

After correcting the sign-in identity and workspace assignment, start a new Genie Code session and test again. If the same error remains, this requires a Databricks-side account/workspace entitlement or provisioning check rather than another change in User Management.

When contacting Databricks, include:
Databricks account ID
Workspace ID and workspace URL
Azure region
The exact identity/UPN used to sign in
Approximate time of the failure and a screenshot of the error
Confirmation that the user identity and workspace assignment were checked

Reference: Genie Code on Azure Databricks.

3. Help Center access is separate from workspace administration

The message stating that the user does not have access to Help Center is not necessarily caused by missing workspace permissions. Help Center access depends on the organization’s Databricks support entitlement and whether the exact email address is registered as an authorized support contact.

The customer should:

Sign in to Help Center with the exact email associated with the Databricks support entitlement.
If mondayhenry31@gmail.com is not the authorized support contact, ask the account owner or support administrator to add the correct email address.
If this is a course, trial, or learning environment without a support contract, use the course support channel or Databricks Community; Help Center case access may not be included.
If access is still denied, email help@databricks.com with the account ID, workspace ID, workspace URL, Azure region, login identity, and the Help Center error.

Reference: Azure Databricks support.

Summary

A separate Azure subscription is not normally required just to use the workspace or assign Databricks administrator roles. However, the screenshots suggest that the account has an identity/role mismatch that should be corrected first. If the Genie Code error continues after using the canonical identity, Databricks Support must verify the workspace’s Genie Code daily allowance and provisioning. Help Center access is a separate support entitlement from workspace administration.

WORKSPACE HAS NO DAILY TOKEN ALLOWANCE

HOguns — Tue, 29 Sep 2026 13:08:28 GMT

Hello Team,

When I try to debug and troubleshoot this error: "This workspace has no daily token allowance for the assistant. Ask your workspace admin to enable it, then try again." I am the Admin, but I still don't see any way to resolve this issue, I have tried creating a budget with the product genie, I have also navigated to the User Management and on there as shown in on of the snapshot the users and the Admin still I keep hitting that error prompt when I use it alongside a notebook in the workspace.

After chatting with the agent on your website, when I try to access the Help Center, I keep getting this error prompt: "You currently do not have access to Help Center. Please reach out to your admin or send an email to help@databricks.com" for my email address: "mondayhenry31@gmail.com".

Is it that I need a separate subscription from my Azure on Databricks and like a separate Admin access which I can't seem to understand. Please help, as I am trying to go through a hands-on course project, and I am stuck here.

Regards

Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

data_pulse — Tue, 29 Sep 2026 11:32:33 GMT

@qduan

Also do check the backend state before re-creating the assistant. I had tested a similar failure path and saw the assistant remain in CREATING while the file source stayed in UPDATING, with no error_info surfaced yet.

Few useful checks:

import requests req = requests.get( f"https://{workspace_url}/api/2.1/{assistant_name}", headers=headers ) print(req.json()) req = requests.get( f"https://{workspace_url}/api/2.1/{assistant_name}/knowledge-sources", headers=headers ) print(req.json())

Look for:
- assistant state (CREATING, ACTIVE, FAILED)
- error_info
- source state (UPDATING, UPDATED, FAILED_UPDATE)
- knowledge_cutoff_time

Also check the Audit Log:

SELECT event_time, action_name, response.status_code, response.error_message, request_params FROM system.access.audit WHERE service_name = 'knowledgeAssistant' AND event_time >= current_timestamp() - INTERVAL 1 HOUR ORDER BY event_time DESC

In my case when I tested, the source stayed stuck in UPDATING and the assistant in CREATING even though the API GET calls returned 200 and no later sync error appeared in the audit log.

So if the UI says the agent cannot load, checking the assistant/source states through the API is useful because the backend may be stuck in provisioning without surfacing a clear failure event yet.

Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

ThiamLee — Tue, 29 Sep 2026 09:35:26 GMT

Interesting issue—especially the index compatibility part. Hopefully there’s a supported way to recover the agent without re-indexing 2,000+ files.

Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

balajij8 — Tue, 29 Sep 2026 09:19:10 GMT

@qduan

Genie is right regarding the embedding models. When Knowledge Assistant ingests files directly, it creates a managed, internal index using databricks-agent-bricks-embedding-v1. Because that model isn't on the supported list for external AI Search indexes (databricks-gte-large-en, databricks-bge-large-en, or databricks-qwen3-embedding-0-6b), it won't appear as an available source for a new assistant as its a design restriction. There is not a supported way to use that auto generated internal index and repurpose it for new assistants. You may prepare an index and embed the files using one of those three supported models if you plan to use index as an source.

For the load agent error, its likely a platform side issue on the agent's serving endpoint and not due to misconfigurations at your end. You can check - Serving in the platform and verify if the agent's endpoint exists and its status currently. If inference table is enabled, you can run a quick query filtering on status_code to get the exact error details if available. You can inspect the raw metadata using the SDK to see the state the backend agent

Given the volume of 2000+ mixed files and the likelihood of a service side issue during creation, you can reach Databricks support. Share them workspace URL, the creation time and the agent name and they can check it.

Recreating the assistant and reingesting the files is the path forward since that internal index cannot be used.

Re: Salesforce Hosted MCP + UC HTTP connection — stops working after ~1 hour?

iankizer — Tue, 29 Sep 2026 01:04:58 GMT

Exact error: HTTP 401, JSON-RPC -32006, "You're not signed in"

Salesforce Hosted MCP + UC HTTP connection — stops working after ~1 hour?

iankizer — Mon, 28 Sep 2026 19:30:40 GMT

We're wiring Salesforce Hosted MCP (read-only SObject server on api.salesforce.com) through a Unity Catalog HTTP connection and an MCP service, using OAuth User-to-Machine Per User.

Login on the MCP service works and tools respond at first. After about an hour, MCP calls fail until we log in again on the service.

On the HTTP connection, access token expiration shows “Not provided by provider.” Other OAuth HTTP connections in the same workspace (different vendors) show an expiration and don’t hit this hourly cliff.

Salesforce side is an External Client App with mcp_api / refresh_token, PKCE, and JWT-based access tokens (as in Salesforce’s Hosted MCP docs). Callback uses the standard Databricks /login/oauth/http.html redirect.

Has anyone got this combo stable past ~1 hour?
Is missing expires_in on Salesforce token responses a known issue for UC / AI Gateway refresh? Any connection settings that actually help (token exchange method, client secret on refresh, etc.)—without turning off JWT on the ECA?

Has anyone seen this sort of pattern and what fixed it (or if you had to escalate to support)? Or, if you've configured a Salesforce MCP in UC AI Gateway, how do have your External Client App set up so this refresh works properply

Thanks.

Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

Louis_Frolio — Mon, 28 Sep 2026 16:59:34 GMT

Hello @qduan , I did some digging and here is what I found.

Genie has the embedding piece right. The Knowledge Assistant docs list exactly three supported models for a bring-your-own AI Search index: databricks-gte-large-en, databricks-bge-large-en, and databricks-qwen3-embedding-0-6b. An index built automatically from a files source uses a Databricks-managed default model that isn't on that list, so it won't show up in the picker for a new assistant. That's documented behavior, not a bug on your end. I couldn't find anything public that promises reuse of that generated index without re-embedding, so I'd treat it as unsupported today.

That restriction doesn't explain the "We could not load the agent" error, though. Those are two separate problems, and the load failure needs actual evidence. On logs and diagnosis:

Query the audit log system table for service_name = knowledgeAssistant. Assistant and knowledge-source create, update, and sync events are logged there, so a failed or stuck sync should be visible.
Call the Knowledge Assistants API (SDK or REST) to get the agent and list its sources. If the API returns a status, you've bypassed whatever the UI is choking on.
In Catalog Explorer, check the generated index and its AI Search endpoint. Under Serving, check the agent's endpoint. Either one offline or failed will break the agent page.
Confirm the original volume still exists and your user still has access to it.
Open browser dev tools, reload the agent page, and grab the failing request and response from the Network tab. That payload is far more specific than the UI message.

Then open a Databricks Support case with the assistant ID, workspace ID, region, failure time, and whatever the audit log and network call gave you. A generic load error alone isn't enough to tell a permissions problem from an ingestion or service-side one. Don't delete the assistant or its source while this is open. Deleting an assistant removes everything associated with it from default storage.

If support can't revive it, you're looking at re-indexing, either a new assistant from the same files or your own AI Search index using one of the three supported models. The upside of building your own is that multiple assistants can share it and it updates automatically with no manual sync. One thing to check with 2000+ mixed files: anything over 100 MB, or over 500 pages for PDF, DOC, and PPT (each slide counts as a page), is skipped during ingestion. If the original build tripped on something in that pile, it could be part of the story.

References:

Knowledge Assistant: https://learn.microsoft.com/en-us/azure/databricks/agents/agent-bricks/knowledge-assistant
Create AI Search endpoints and indexes: https://learn.microsoft.com/en-us/azure/databricks/ai-search/create-ai-search
Audit log reference (knowledgeAssistant events): https://learn.microsoft.com/en-us/azure/databricks/admin/account-settings/audit-logs
Knowledge Assistants API: https://docs.databricks.com/api/workspace/knowledgeassistants
Databricks Help Center: https://help.databricks.com/s/

Regards, Louis.

Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

qduan — Mon, 28 Sep 2026 12:00:37 GMT

Hi Databricks Community,

I created a Knowledge Assistant through Databricks → Agents, using files as the knowledge source. The embedding and indexing were handled automatically during setup.

Since then, I have been unable to open the assistant. The UI displays:

We could not load the agent
Please try again later, or contact support.

Workaround attempted

To continue working, I tried creating another Knowledge Assistant and selecting the existing AI Search index as its knowledge source. However, the index did not appear as an available option.

I asked Genie to investigate. It reported that the automatically created index, [index_name], uses databricks-agent-bricks-embedding-v1, while Knowledge Assistant supports indexes using only:

databricks-gte-large-en
databricks-bge-large-en
databricks-qwen3-embedding-0-6b

This is Genie's explanation; I have not independently confirmed that this restriction is the cause.

Expected behavior

The original Knowledge Assistant should remain accessible after indexing. Alternatively, I would expect to be able to reuse its generated index in a new assistant without embedding and indexing the same files again.

Questions

How can I diagnose and resolve the “We could not load the agent” error? Are there logs I can check?
Is an index using databricks-agent-bricks-embedding-v1 intentionally unavailable as a source for another Knowledge Assistant?
Is there a supported way to recover the original assistant or reuse its existing index without re-embedding the source files?

Environment

Cloud provider / region: Azure/EMEA
Approximate creation time and time zone: 11-Sep, around 12am Amsterdam time
Source file types and approximate volume: more than 2000 files, mixed types, PDFs, pptx, etc.

Any guidance on recovery or index reuse would be appreciated.