Hi all,
We're hitting a sudden failure with Databrick's built-in python sandbox system.ai.python_exec function and want to check whether others are seeing the same thing, since it looks region-specific.
Environment:
- Cloud: Azure Databricks
- Region: West US
- Both interactive notebooks and serverless
This was working reliably until Wednesday, May 27 at ~4:00 PM EST. Immediately after that, calls to system.ai.python_exec began failing. Nothing changed on our side. No permission edits, no code changes, no runtime upgrade.
Error:
{"ts": "2026-05-29 17:30:20.716", "level": "ERROR", "logger": "pyspark.sql.connect.logging", "msg": "GRPC Error received", "context": {}, "exception": {"class": "_MultiThreadedRendezvous", "msg": "<_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.INTERNAL
details = "[UNRESOLVED_ROUTINE] Cannot resolve routine `system`.`ai`.`python_exec` on search path [`system`.`session`, `system`.`builtin`, `system`.`ai`, `system`.`default`].
Verify the spelling of `system`.`ai`.`python_exec`, check that the routine exists, and confirm you have `USE` privilege on the catalog and schema, and EXECUTE on the routine.
You can see that system.ai is in the search path shown in the error, so the resolver is looking in the right place, it just can't find the routine.
What we've already verified:
- USE on the system catalog and ai schema, and EXECUTE on the routine, all permissions are intact and unchanged.
- The fully qualified name system.ai.python_exec is correct (no typo, no missing qualifier).
- No change on our end coincided with the failure time.
Workaround that works for us:
Recreating the same python_exec function in a non-system catalog and calling it there works fine in the same region. So the underlying compute/execution path is healthy. The issue appears specific to resolving/executing the built-in copy under system.ai.
Question for the community / Databricks
- Is anyone else seeing system.ai.python_exec fail to resolve on Azure West US (or other regions), across notebooks and serverless?
- Is this a known regional issue or a recent change to the system.ai schema?
Any confirmation or pointers appreciated. Happy to share more details with Databricks staff if helpful.
Thank you!
