Louis_Frolio
Databricks Employee
Databricks Employee

Monday

Hello @qduan , I did some digging and here is what I found.

Genie has the embedding piece right. The Knowledge Assistant docs list exactly three supported models for a bring-your-own AI Search index: databricks-gte-large-en, databricks-bge-large-en, and databricks-qwen3-embedding-0-6b. An index built automatically from a files source uses a Databricks-managed default model that isn't on that list, so it won't show up in the picker for a new assistant. That's documented behavior, not a bug on your end. I couldn't find anything public that promises reuse of that generated index without re-embedding, so I'd treat it as unsupported today.

That restriction doesn't explain the "We could not load the agent" error, though. Those are two separate problems, and the load failure needs actual evidence. On logs and diagnosis:

  1. Query the audit log system table for service_name = knowledgeAssistant. Assistant and knowledge-source create, update, and sync events are logged there, so a failed or stuck sync should be visible.
  2. Call the Knowledge Assistants API (SDK or REST) to get the agent and list its sources. If the API returns a status, you've bypassed whatever the UI is choking on.
  3. In Catalog Explorer, check the generated index and its AI Search endpoint. Under Serving, check the agent's endpoint. Either one offline or failed will break the agent page.
  4. Confirm the original volume still exists and your user still has access to it.
  5. Open browser dev tools, reload the agent page, and grab the failing request and response from the Network tab. That payload is far more specific than the UI message.

Then open a Databricks Support case with the assistant ID, workspace ID, region, failure time, and whatever the audit log and network call gave you. A generic load error alone isn't enough to tell a permissions problem from an ingestion or service-side one. Don't delete the assistant or its source while this is open. Deleting an assistant removes everything associated with it from default storage.

If support can't revive it, you're looking at re-indexing, either a new assistant from the same files or your own AI Search index using one of the three supported models. The upside of building your own is that multiple assistants can share it and it updates automatically with no manual sync. One thing to check with 2000+ mixed files: anything over 100 MB, or over 500 pages for PDF, DOC, and PPT (each slide counts as a page), is skipped during ingestion. If the original build tripped on something in that pile, it could be part of the story.

References:

 

Regards, Louis.

0 Kudos