All Generative AI posts

Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

data_pulse — Tue, 29 Sep 2026 11:32:33 GMT

Also do check the backend state before re-creating the assistant. I had tested a similar failure path and saw the assistant remain in CREATING while the file source stayed in UPDATING, with no error_info surfaced yet.

Few useful checks:

import requests req = requests.get( f"https://{workspace_url}/api/2.1/{assistant_name}", headers=headers ) print(req.json()) req = requests.get( f"https://{workspace_url}/api/2.1/{assistant_name}/knowledge-sources", headers=headers ) print(req.json())

Look for:
- assistant state (CREATING, ACTIVE, FAILED)
- error_info
- source state (UPDATING, UPDATED, FAILED_UPDATE)
- knowledge_cutoff_time

Also check the Audit Log:

SELECT event_time, action_name, response.status_code, response.error_message, request_params FROM system.access.audit WHERE service_name = 'knowledgeAssistant' AND event_time >= current_timestamp() - INTERVAL 1 HOUR ORDER BY event_time DESC

In my case when I tested, the source stayed stuck in UPDATING and the assistant in CREATING even though the API GET calls returned 200 and no later sync error appeared in the audit log.

So if the UI says the agent cannot load, checking the assistant/source states through the API is useful because the backend may be stuck in provisioning without surfacing a clear failure event yet.

Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

ThiamLee — Tue, 29 Sep 2026 09:35:26 GMT

Interesting issue—especially the index compatibility part. Hopefully there’s a supported way to recover the agent without re-indexing 2,000+ files.

Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

balajij8 — Tue, 29 Sep 2026 09:19:10 GMT

@qduan

Genie is right regarding the embedding models. When Knowledge Assistant ingests files directly, it creates a managed, internal index using databricks-agent-bricks-embedding-v1. Because that model isn't on the supported list for external AI Search indexes (databricks-gte-large-en, databricks-bge-large-en, or databricks-qwen3-embedding-0-6b), it won't appear as an available source for a new assistant as its a design restriction. There is not a supported way to use that auto generated internal index and repurpose it for new assistants. You may prepare an index and embed the files using one of those three supported models if you plan to use index as an source.

For the load agent error, its likely a platform side issue on the agent's serving endpoint and not due to misconfigurations at your end. You can check - Serving in the platform and verify if the agent's endpoint exists and its status currently. If inference table is enabled, you can run a quick query filtering on status_code to get the exact error details if available. You can inspect the raw metadata using the SDK to see the state the backend agent

Given the volume of 2000+ mixed files and the likelihood of a service side issue during creation, you can reach Databricks support. Share them workspace URL, the creation time and the agent name and they can check it.

Recreating the assistant and reingesting the files is the path forward since that internal index cannot be used.

Re: Salesforce Hosted MCP + UC HTTP connection — stops working after ~1 hour?

iankizer — Tue, 29 Sep 2026 01:04:58 GMT

Exact error: HTTP 401, JSON-RPC -32006, "You're not signed in"

Salesforce Hosted MCP + UC HTTP connection — stops working after ~1 hour?

iankizer — Mon, 28 Sep 2026 19:30:40 GMT

We're wiring Salesforce Hosted MCP (read-only SObject server on api.salesforce.com) through a Unity Catalog HTTP connection and an MCP service, using OAuth User-to-Machine Per User.

Login on the MCP service works and tools respond at first. After about an hour, MCP calls fail until we log in again on the service.

On the HTTP connection, access token expiration shows “Not provided by provider.” Other OAuth HTTP connections in the same workspace (different vendors) show an expiration and don’t hit this hourly cliff.

Salesforce side is an External Client App with mcp_api / refresh_token, PKCE, and JWT-based access tokens (as in Salesforce’s Hosted MCP docs). Callback uses the standard Databricks /login/oauth/http.html redirect.

Has anyone got this combo stable past ~1 hour?
Is missing expires_in on Salesforce token responses a known issue for UC / AI Gateway refresh? Any connection settings that actually help (token exchange method, client secret on refresh, etc.)—without turning off JWT on the ECA?

Has anyone seen this sort of pattern and what fixed it (or if you had to escalate to support)? Or, if you've configured a Salesforce MCP in UC AI Gateway, how do have your External Client App set up so this refresh works properply

Thanks.

Re: Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

Louis_Frolio — Mon, 28 Sep 2026 16:59:34 GMT

Hello @qduan , I did some digging and here is what I found.

Genie has the embedding piece right. The Knowledge Assistant docs list exactly three supported models for a bring-your-own AI Search index: databricks-gte-large-en, databricks-bge-large-en, and databricks-qwen3-embedding-0-6b. An index built automatically from a files source uses a Databricks-managed default model that isn't on that list, so it won't show up in the picker for a new assistant. That's documented behavior, not a bug on your end. I couldn't find anything public that promises reuse of that generated index without re-embedding, so I'd treat it as unsupported today.

That restriction doesn't explain the "We could not load the agent" error, though. Those are two separate problems, and the load failure needs actual evidence. On logs and diagnosis:

Query the audit log system table for service_name = knowledgeAssistant. Assistant and knowledge-source create, update, and sync events are logged there, so a failed or stuck sync should be visible.
Call the Knowledge Assistants API (SDK or REST) to get the agent and list its sources. If the API returns a status, you've bypassed whatever the UI is choking on.
In Catalog Explorer, check the generated index and its AI Search endpoint. Under Serving, check the agent's endpoint. Either one offline or failed will break the agent page.
Confirm the original volume still exists and your user still has access to it.
Open browser dev tools, reload the agent page, and grab the failing request and response from the Network tab. That payload is far more specific than the UI message.

Then open a Databricks Support case with the assistant ID, workspace ID, region, failure time, and whatever the audit log and network call gave you. A generic load error alone isn't enough to tell a permissions problem from an ingestion or service-side one. Don't delete the assistant or its source while this is open. Deleting an assistant removes everything associated with it from default storage.

If support can't revive it, you're looking at re-indexing, either a new assistant from the same files or your own AI Search index using one of the three supported models. The upside of building your own is that multiple assistants can share it and it updates automatically with no manual sync. One thing to check with 2000+ mixed files: anything over 100 MB, or over 500 pages for PDF, DOC, and PPT (each slide counts as a page), is skipped during ingestion. If the original build tripped on something in that pile, it could be part of the story.

References:

Knowledge Assistant: https://learn.microsoft.com/en-us/azure/databricks/agents/agent-bricks/knowledge-assistant
Create AI Search endpoints and indexes: https://learn.microsoft.com/en-us/azure/databricks/ai-search/create-ai-search
Audit log reference (knowledgeAssistant events): https://learn.microsoft.com/en-us/azure/databricks/admin/account-settings/audit-logs
Knowledge Assistants API: https://docs.databricks.com/api/workspace/knowledgeassistants
Databricks Help Center: https://help.databricks.com/s/

Regards, Louis.

Knowledge Assistant fails to load after indexing; generated AI Search index cannot be reused

qduan — Mon, 28 Sep 2026 12:00:37 GMT

Hi Databricks Community,

I created a Knowledge Assistant through Databricks → Agents, using files as the knowledge source. The embedding and indexing were handled automatically during setup.

Since then, I have been unable to open the assistant. The UI displays:

We could not load the agent
Please try again later, or contact support.

Workaround attempted

To continue working, I tried creating another Knowledge Assistant and selecting the existing AI Search index as its knowledge source. However, the index did not appear as an available option.

I asked Genie to investigate. It reported that the automatically created index, [index_name], uses databricks-agent-bricks-embedding-v1, while Knowledge Assistant supports indexes using only:

databricks-gte-large-en
databricks-bge-large-en
databricks-qwen3-embedding-0-6b

This is Genie's explanation; I have not independently confirmed that this restriction is the cause.

Expected behavior

The original Knowledge Assistant should remain accessible after indexing. Alternatively, I would expect to be able to reuse its generated index in a new assistant without embedding and indexing the same files again.

Questions

How can I diagnose and resolve the “We could not load the agent” error? Are there logs I can check?
Is an index using databricks-agent-bricks-embedding-v1 intentionally unavailable as a source for another Knowledge Assistant?
Is there a supported way to recover the original assistant or reuse its existing index without re-embedding the source files?

Environment

Cloud provider / region: Azure/EMEA
Approximate creation time and time zone: 11-Sep, around 12am Amsterdam time
Source file types and approximate volume: more than 2000 files, mixed types, PDFs, pptx, etc.

Any guidance on recovery or index reuse would be appreciated.

Re: Genie Agent response Export to PDF via API

GunaR — Mon, 28 Sep 2026 10:52:45 GMT

Thank you for the detailed analysis and input.

I tried with List Conversations items endpoint, weasyprint for MD to HTML and on the agent request with Enable_viz:False.
Here are my observations,
1) Overall the output is not similar to export to excel.
2) The Table layout is broken and not fit in a page.
3) The symbols are missing or substituted by different characters ( Ex - Green tick).

Any roadmap to get the Export to PDF simulation as an endpoint to get the same output via API.

What Are the Key Considerations for Building AI-Powered Applications?

tarunnagar — Mon, 28 Sep 2026 10:03:22 GMT

Hi everyone,

I’m exploring how teams are approaching AI-powered application development and would like to understand some practical considerations when moving from an AI prototype to a production application.

For teams using Databricks, I’m particularly interested in how you approach:

Connecting application data with AI or LLM-based features
Choosing between different model-serving approaches
Using vector search and RAG for application-specific knowledge
Evaluating AI responses before deploying an application
Managing latency and scalability for AI-powered apps
Adding security, governance, and guardrails to AI applications
Monitoring AI applications after deployment

For those who have built AI applications with Databricks, what were the biggest challenges you encountered, and which architecture or practices helped you address them?

I’d especially appreciate examples from real-world production applications.

Re: Genie Agent SDK visualization

Gecofer — Sun, 27 Sep 2026 18:11:12 GMT

Great, thanks for confirming!

Since the signature now includes enable_visualization, it looks like the SDK itself was correct and the issue was most likely related to the Python process still using a previously loaded version/module.

Restarting the Python process after upgrading or changing the SDK is therefore an important step here.

Glad it is working now!

Re: DeepSeek V4.1 Flash returns workspace ITPM rate-limit error on fresh/idle workspace

Universeb — Sat, 26 Sep 2026 00:00:56 GMT

Thanks. I checked that setting.

"Enforce data processing within workspace Geography for Designated Services"

was already disabled when I reproduced the issue, so cross-Geo processing was allowed.

DeepSeek V4.1 Flash still returned the same ITPM 429 in both AI Playground

and the REST API.

REST reproduction:

- Cloud: AWS

- Region: Seoul (ap-northeast-2)

- Workspace tier: Enterprise

- HTTP status: 429

- UTC timestamp: 2026-09-25 22:54:35 UTC

- Request ID: b03cbf14-6dee-49d0-abfe-2e70e9450005

Response:

REQUEST_LIMIT_EXCEEDED: Exceeded workspace input tokens per minute rate limit for

databricks-deepseek-v4-1-flash.

No limit, current, or retry-after values were returned.

Re: DeepSeek V4.1 Flash returns workspace ITPM rate-limit error on fresh/idle workspace

ivanvyd — Fri, 25 Sep 2026 23:21:55 GMT

Thanks, @Universeb. One region-specific detail: Databricks lists DeepSeek V4.1 Flash in ap-northeast-2 as requiring cross-Geo routing, with availability also dependent on GPU capacity.

Could your account admin check Account Console > Workspaces > your workspace > Security and compliance? "Enforce data processing within workspace Geography for Designated Services" needs to be off to allow cross-Geo processing. Please only change it if your data-residency policy permits.

Re: DeepSeek V4.1 Flash returns workspace ITPM rate-limit error on fresh/idle workspace

Universeb — Fri, 25 Sep 2026 22:56:55 GMT

Thanks. I reproduced the issue again via the REST API.

Environment:
- Cloud: AWS
- Region: Seoul (ap-northeast-2)
- Workspace tier: Enterprise

Test:
- Model: databricks-deepseek-v4-1-flash
- Prompt: "hi"
- No concurrent workload against this model
- HTTP status: 429
- UTC timestamp: 2026-09-25 22:54:35 UTC
- Request ID: b03cbf14-6dee-49d0-abfe-2e70e9450005

Raw response body:

{"error_code":"REQUEST_LIMIT_EXCEEDED","message":"REQUEST_LIMIT_EXCEEDED: Exceeded workspace input tokens per minute rate limit for databricks-deepseek-v4-1-flash. Please use a provisioned throughput Foundation Model APIs endpoint for a higher rate limit."}

Relevant response headers:

HTTP/1.1 429 Too Many Requests
date: Fri, 25 Sep 2026 22:54:35 GMT
x-request-id: b03cbf14-6dee-49d0-abfe-2e70e9450005

There were no limit/current/retry-after values returned in either the response body or the relevant response headers.

The REST request used only a minimal "hi" prompt. The same issue also reproduces in AI Playground, while other Databricks-hosted foundation models work normally.

One additional detail that may be relevant: DeepSeek V4.1 Flash previously worked successfully for me, but later started consistently returning this workspace ITPM error.

Please let me know if there is any other diagnostic information I can provide.

Re: DeepSeek V4.1 Flash returns workspace ITPM rate-limit error on fresh/idle workspace

ivanvyd — Fri, 25 Sep 2026 22:22:14 GMT

@Universeb could you add your cloud, region and workspace tier, please? The published rate limits depend on the tier, so they don't establish your workspace's assigned quota.

For one failed REST call, please include the UTC timestamp, any request ID, and sanitised error details -particularly limit and current, if returned. Those fields would help distinguish usage exhaustion from an unexpectedly low allowance. I'd ask Databricks to check that before switching to provisioned throughput.

Re: Genie Higher Environment deployment using DAB

ivanvyd — Fri, 25 Sep 2026 22:08:47 GMT

@karuppusamy one practical addition to @Ashwin_DSA CLI steps: I'd explicitly separent_path on the genie_spaces resource to an existing workspace folder outside the Git folder, writable by your deployment identity.

These two paths serve different purposes: file_path points to the exported JSON definition in your Azure DevOps checkout; parent_path controls where the deployed Genie Agent is registered. Both are documented in the Genie bundle resource reference⁠.

Keep the YAML and JSON definition in source control, but keep the native Genie Agent out of the workspace Git folder. The Git-folder restriction⁠ concerns the native agent asset; exporting its definition and deploying it through the CLI is a separate operation.

Re: Both Databricks Apps Returning 502 Bad Gateway

Jeneive — Fri, 25 Sep 2026 19:26:54 GMT

Thank you for the detailed guidance. The issue has been resolved, and both applications are now working fine.

My assumption is that the issue was related to the application authentication check. The apps are now working independently without requiring authentication.

I appreciate your help and the troubleshooting suggestions.

Re: Genie Higher Environment deployment using DAB

Ashwin_DSA — Fri, 25 Sep 2026 13:57:58 GMT

Hi @karuppusamy,

Thanks for sharing the error snapshot. What you are seeing is expected behaviour rather than a bundle misconfiguration. Genie Agents are an unsupported asset type for commits in Databricks Git folders, so the workspace Git integration cannot sync the .geniespace.json. That is why your notebooks and dashboards go through fine, but the Genie does not.

The fix is to keep the Genie out of the workspace Git folder sync path and let the Databricks CLI deploy it from your Azure DevOps checkout through the API.

There is no UI download for a Genie Agent, so generate it with the bundle generate genie-space command.

databricks bundle generate genie-space \ --existing-id <your-genie-space-id> \ --key inventory_genie

This writes the resource YAML and a src/*.geniespace.json into your bundle. If you prefer the REST route instead, call the Genie Space API with GET /api/2.0/genie/spaces/{space_id}?include_serialized_space=true and save the serialized_space payload.

Keep that .geniespace.json in your Azure DevOps repo alongside databricks.yml, not inside a Databricks workspace Git folder. Set engine: direct in the bundle, which the genie_spaces resource requires.

Deploy from the pipeline with the CLI, which reads the file from the agent filesystem and pushes it via the API. This slots straight into the standard Azure DevOps CI/CD flow for bundles.

databricks bundle validate --target $(BUNDLE_TARGET) databricks bundle deploy --target $(BUNDLE_TARGET)

A quick checklist before you retry.

Databricks CLI 1.3.0 or later, latest preferred, which you can confirm with databricks --version.
engine: direct set in databricks.yml, per the direct deployment engine docs.
The .geniespace.json lives in the external repo and is never synced through a Databricks Git folder.
The warehouse id and any catalog or schema the Genie references exist in the target workspace.

Once you move to CLI-driven deploys from the pipeline instead of Git folder sync, the Genie will land in QA and prod cleanly.

DeepSeek V4.1 Flash returns workspace ITPM rate-limit error on fresh/idle workspace

Universeb — Fri, 25 Sep 2026 13:11:52 GMT

Hi, I am unable to use the Databricks-hosted DeepSeek V4.1 Flash model. Even a minimal prompt such as "hi" immediately fails with: REQUEST_LIMIT_EXCEEDED: Exceeded workspace input tokens per minute rate limit for databricks-deepseek-v4-1-flash. Please use a provisioned throughput Foundation Model APIs endpoint for a higher rate limit. This does not appear to be caused by actual token usage. I have reproduced the issue under the following conditions: - A fresh/idle workspace - A fresh AI Playground session - A prompt containing only "hi" - Direct REST API requests - After creating a completely new workspace - Other Databricks-hosted foundation models work normally Only DeepSeek V4.1 Flash is affected. Could a Databricks employee please verify whether there is a model-specific effective ITPM quota, entitlement, or backend serving allocation issue affecting: databricks-deepseek-v4-1-flash I previously opened Databricks Support case #01025570, but I was redirected to the Community because the account does not have a paid support contract. I have attached a screenshot showing the error from AI Playground. Thank you.

Re: Genie Agent refuses to add scalar SQL function from Unity Catalog

ivanvyd — Fri, 25 Sep 2026 12:44:26 GMT

Thanks for reproducing this, @ThomazNeto. @charl-p-botha, I'd try Configure > Examples > Add on your existing agent before recreating it. That matches the Azure documentation's route for adding SQL functions.

One small testing detail: your screenshot's function description already includes 42, so Genie returning 42 wouldn't by itself confirm that it executed the function. After saving, check the function / query details too: Show code for a SQL response, or Show more for a trusted-function response in Chat mode.

If Examples still rejects it - or saves successfully but the function cannot be called - that's the specific distinction I'd take to your account team.

Re: Genie Agent refuses to add scalar SQL function from Unity Catalog

ThomazNeto — Fri, 25 Sep 2026 12:10:46 GMT

Good news, Charl: I think we found it. I reproduced your error on a separate Azure workspace today and then tried the other path, which works.

Configure > Sources > Add > SQL function rejects scalar functions with "You must select a function that returns a table". I tried a scalar with no arguments and one with a DATE parameter, and both were blocked. A table function wrapping the same value saved without problems. This is the "provide verified answers" dialog, and it only accepts table functions.
Configure > Examples > Add accepts the same scalar function. It shows the definition along with the Parameters and Usage Guidance sections, and saves fine.

This matches the docs, which only describe adding functions through the Examples tab: "Use the Examples tab to add... SQL functions". What makes it confusing is that the Sources menu also lists "SQL function", so it looks like the natural place to add one.
https://docs.databricks.com/aws/en/genie-agents/tune-quality#how-does-genie-use-sql-functions

So you don't need the table wrapper. Add the scalar through Examples and fill in the Usage Guidance so Genie knows when to call it. It may still be worth flagging to the account team that the Sources path rejects scalar functions with an error that suggests they aren't supported at all.