topic Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused in Generative AI

Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

qduan — Mon, 28 Sep 2026 12:00:37 GMT

Hi Databricks Community,

I created a Knowledge Assistant through Databricks → Agents, using files as the knowledge source. The embedding and indexing were handled automatically during setup.

Since then, I have been unable to open the assistant. The UI displays:

We could not load the agent
Please try again later, or contact support.

Workaround attempted

To continue working, I tried creating another Knowledge Assistant and selecting the existing AI Search index as its knowledge source. However, the index did not appear as an available option.

I asked Genie to investigate. It reported that the automatically created index, [index_name], uses databricks-agent-bricks-embedding-v1, while Knowledge Assistant supports indexes using only:

databricks-gte-large-en
databricks-bge-large-en
databricks-qwen3-embedding-0-6b

This is Genie's explanation; I have not independently confirmed that this restriction is the cause.

Expected behavior

The original Knowledge Assistant should remain accessible after indexing. Alternatively, I would expect to be able to reuse its generated index in a new assistant without embedding and indexing the same files again.

Questions

How can I diagnose and resolve the “We could not load the agent” error? Are there logs I can check?
Is an index using databricks-agent-bricks-embedding-v1 intentionally unavailable as a source for another Knowledge Assistant?
Is there a supported way to recover the original assistant or reuse its existing index without re-embedding the source files?

Environment

Cloud provider / region: Azure/EMEA
Approximate creation time and time zone: 11-Sep, around 12am Amsterdam time
Source file types and approximate volume: more than 2000 files, mixed types, PDFs, pptx, etc.

Any guidance on recovery or index reuse would be appreciated.

Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

Louis_Frolio — Mon, 28 Sep 2026 16:59:34 GMT

Hello @qduan , I did some digging and here is what I found.

Genie has the embedding piece right. The Knowledge Assistant docs list exactly three supported models for a bring-your-own AI Search index: databricks-gte-large-en, databricks-bge-large-en, and databricks-qwen3-embedding-0-6b. An index built automatically from a files source uses a Databricks-managed default model that isn't on that list, so it won't show up in the picker for a new assistant. That's documented behavior, not a bug on your end. I couldn't find anything public that promises reuse of that generated index without re-embedding, so I'd treat it as unsupported today.

That restriction doesn't explain the "We could not load the agent" error, though. Those are two separate problems, and the load failure needs actual evidence. On logs and diagnosis:

Query the audit log system table for service_name = knowledgeAssistant. Assistant and knowledge-source create, update, and sync events are logged there, so a failed or stuck sync should be visible.
Call the Knowledge Assistants API (SDK or REST) to get the agent and list its sources. If the API returns a status, you've bypassed whatever the UI is choking on.
In Catalog Explorer, check the generated index and its AI Search endpoint. Under Serving, check the agent's endpoint. Either one offline or failed will break the agent page.
Confirm the original volume still exists and your user still has access to it.
Open browser dev tools, reload the agent page, and grab the failing request and response from the Network tab. That payload is far more specific than the UI message.

Then open a Databricks Support case with the assistant ID, workspace ID, region, failure time, and whatever the audit log and network call gave you. A generic load error alone isn't enough to tell a permissions problem from an ingestion or service-side one. Don't delete the assistant or its source while this is open. Deleting an assistant removes everything associated with it from default storage.

If support can't revive it, you're looking at re-indexing, either a new assistant from the same files or your own AI Search index using one of the three supported models. The upside of building your own is that multiple assistants can share it and it updates automatically with no manual sync. One thing to check with 2000+ mixed files: anything over 100 MB, or over 500 pages for PDF, DOC, and PPT (each slide counts as a page), is skipped during ingestion. If the original build tripped on something in that pile, it could be part of the story.

References:

Knowledge Assistant: https://learn.microsoft.com/en-us/azure/databricks/agents/agent-bricks/knowledge-assistant
Create AI Search endpoints and indexes: https://learn.microsoft.com/en-us/azure/databricks/ai-search/create-ai-search
Audit log reference (knowledgeAssistant events): https://learn.microsoft.com/en-us/azure/databricks/admin/account-settings/audit-logs
Knowledge Assistants API: https://docs.databricks.com/api/workspace/knowledgeassistants
Databricks Help Center: https://help.databricks.com/s/

Regards, Louis.

Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

balajij8 — Tue, 29 Sep 2026 09:19:10 GMT

@qduan

Genie is right regarding the embedding models. When Knowledge Assistant ingests files directly, it creates a managed, internal index using databricks-agent-bricks-embedding-v1. Because that model isn't on the supported list for external AI Search indexes (databricks-gte-large-en, databricks-bge-large-en, or databricks-qwen3-embedding-0-6b), it won't appear as an available source for a new assistant as its a design restriction. There is not a supported way to use that auto generated internal index and repurpose it for new assistants. You may prepare an index and embed the files using one of those three supported models if you plan to use index as an source.

For the load agent error, its likely a platform side issue on the agent's serving endpoint and not due to misconfigurations at your end. You can check - Serving in the platform and verify if the agent's endpoint exists and its status currently. If inference table is enabled, you can run a quick query filtering on status_code to get the exact error details if available. You can inspect the raw metadata using the SDK to see the state the backend agent

Given the volume of 2000+ mixed files and the likelihood of a service side issue during creation, you can reach Databricks support. Share them workspace URL, the creation time and the agent name and they can check it.

Recreating the assistant and reingesting the files is the path forward since that internal index cannot be used.

Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

ThiamLee — Tue, 29 Sep 2026 09:35:26 GMT

Interesting issue—especially the index compatibility part. Hopefully there’s a supported way to recover the agent without re-indexing 2,000+ files.

Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

data_pulse — Tue, 29 Sep 2026 11:32:33 GMT

@qduan

Also do check the backend state before re-creating the assistant. I had tested a similar failure path and saw the assistant remain in CREATING while the file source stayed in UPDATING, with no error_info surfaced yet.

Few useful checks:

import requests req = requests.get( f"https://{workspace_url}/api/2.1/{assistant_name}", headers=headers ) print(req.json()) req = requests.get( f"https://{workspace_url}/api/2.1/{assistant_name}/knowledge-sources", headers=headers ) print(req.json())

Look for:
- assistant state (CREATING, ACTIVE, FAILED)
- error_info
- source state (UPDATING, UPDATED, FAILED_UPDATE)
- knowledge_cutoff_time

Also check the Audit Log:

SELECT event_time, action_name, response.status_code, response.error_message, request_params FROM system.access.audit WHERE service_name = 'knowledgeAssistant' AND event_time >= current_timestamp() - INTERVAL 1 HOUR ORDER BY event_time DESC

In my case when I tested, the source stayed stuck in UPDATING and the assistant in CREATING even though the API GET calls returned 200 and no later sync error appeared in the audit log.

So if the UI says the agent cannot load, checking the assistant/source states through the API is useful because the backend may be stuck in provisioning without surfacing a clear failure event yet.