system.ai.python_exec suddenly failing with UNRESOLVED_ROUTINE on Azure (West US) since May 27 ~4pm

[sundarep_cav]

Hi all, 

We're hitting a sudden failure with Databrick's built-in python sandbox system.ai.python_exec function and want to check whether others are seeing the same thing, since it looks region-specific.

Environment:

• Cloud: Azure Databricks
• Region: West US
• Both interactive notebooks and serverless

This was working reliably until Wednesday, May 27 at ~4:00 PM EST. Immediately after that, calls to system.ai.python_exec began failing. Nothing changed on our side. No permission edits, no code changes, no runtime upgrade.

Error:

{"ts": "2026-05-29 17:30:20.716", "level": "ERROR", "logger": "pyspark.sql.connect.logging", "msg": "GRPC Error received", "context": {}, "exception": {"class": "_MultiThreadedRendezvous", "msg": "<_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.INTERNAL
details = "[UNRESOLVED_ROUTINE] Cannot resolve routine `system`.`ai`.`python_exec` on search path [`system`.`session`, `system`.`builtin`, `system`.`ai`, `system`.`default`].
Verify the spelling of `system`.`ai`.`python_exec`, check that the routine exists, and confirm you have `USE` privilege on the catalog and schema, and EXECUTE on the routine.

You can see that system.ai is in the search path shown in the error, so the resolver is looking in the right place, it just can't find the routine.

What we've already verified:

• USE on the system catalog and ai schema, and EXECUTE on the routine, all permissions are intact and unchanged.
• The fully qualified name system.ai.python_exec is correct (no typo, no missing qualifier).
• No change on our end coincided with the failure time.

Workaround that works for us:
Recreating the same python_exec function in a non-system catalog and calling it there works fine in the same region. So the underlying compute/execution path is healthy. The issue appears specific to resolving/executing the built-in copy under system.ai.

Question for the community / Databricks

• Is anyone else seeing system.ai.python_exec fail to resolve on Azure West US (or other regions), across notebooks and serverless?
• Is this a known regional issue or a recent change to the system.ai schema?

Any confirmation or pointers appreciated. Happy to share more details with Databricks staff if helpful.

Thank you!

[emma_s]

Hi, just had a quick look into this and it looks unlikely to be a customer-side permissions issue. system.ai.python_exec is a built-in Databricks function/tool, and there have been region-specific cases where the function was unavailable because it had been disabled or not correctly rolled out in a given region. The fact that an equivalent function works fine in a non-system catalog strongly suggests the execution path is healthy and the issue is specific to the built-in system.ai copy. If you're still getting this problem I'd recommend opening a support ticket with workspace ID, metastore ID, cloud/region, and first-failure timestamp.

Thanks,

Emma

[rm_epam]

Do you have any updates?

I have the same issue AWS Databricks in us-west-2 region. The only thing looks very confusing to me that result of this code:

USE CATALOG system;
USE SCHEMA ai;

SELECT python_exec('print("hello")');

Contains duplicates mention of `system.ai` :

[UNRESOLVED_ROUTINE] Cannot resolve routine `python_exec` on search path [`system`.`session`, `system`.`builtin`, `system`.`ai`, `system`.`ai`].
Verify the spelling of `python_exec`, check that the routine exists, and confirm you have `USE` privilege on the catalog and schema, and EXECUTE on the routine. SQLSTATE: 42883
== SQL (line 37, position 8) ==
SELECT python_exec('print("hello")')