Trigger a Databricks job using sql query
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-26-2025 01:38 AM
Trying to run a SQL call function to trigger a Databricks job and am stuck with an error stating:
"The schema <schema_name>.system cannot be found. Verify the spelling and correctness of the schema and catalog. If you did not qualify the name with a catalog, verify the current_schema() output, or qualify the name with the correct catalog. To tolerate the error on drop use DROP SCHEMA IF EXISTS. SQLSTATE: 42704".
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-26-2025 01:55 AM
The error is due to the non-existence or incorrect reference to the "system" schema under your catalog. Check the correct catalog and schema names where the stored procedure run_job resides, and update the CALL statement accordingly.
CALL test_catalog.system.run_job("load_table")
Verify that the schema "system" truly exists in the catalog test_catalog
If "run_job" is a stored procedure or SQL function, be sure it exists under the correct catalog and schema name.
if you intended to execute a stored procedure or run a job, confirm that it is created in a schema you have access to and specify that schema instead of "system"
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-26-2025 02:16 AM
When you say "Databricks Job", do you mean a custom SQL code you created in an stored procedure or function? Saying this because if you're really trying to run a real Databricks Job (lakeflow), it would be much better to establish another strategy different from that initial one. I mean, create firstly the Databricks Job and then, add type of tasks better for you scenario (https://docs.databricks.com/aws/en/jobs/configure-task#types-of-tasks )