topic Re: Trigger a Databricks job using sql query in Data Engineering

Trigger a Databricks job using sql query

Neo_Reeves — Wed, 26 Nov 2025 09:38:16 GMT

Trying to run a SQL call function to trigger a Databricks job and am stuck with an error stating:
"The schema <schema_name>.system cannot be found. Verify the spelling and correctness of the schema and catalog. If you did not qualify the name with a catalog, verify the current_schema() output, or qualify the name with the correct catalog. To tolerate the error on drop use DROP SCHEMA IF EXISTS. SQLSTATE: 42704".

The query that I am trying to run:

%sql

CALL test_catalog.system.run_job("load_table")

Re: Trigger a Databricks job using sql query

ManojkMohan — Wed, 26 Nov 2025 09:55:52 GMT

@Neo_Reeves

The error is due to the non-existence or incorrect reference to the "system" schema under your catalog. Check the correct catalog and schema names where the stored procedure run_job resides, and update the CALL statement accordingly.

CALL test_catalog.system.run_job("load_table")

Verify that the schema "system" truly exists in the catalog test_catalog

If "run_job" is a stored procedure or SQL function, be sure it exists under the correct catalog and schema name.

if you intended to execute a stored procedure or run a job, confirm that it is created in a schema you have access to and specify that schema instead of "system"

Re: Trigger a Databricks job using sql query

Coffee77 — Wed, 26 Nov 2025 10:16:57 GMT

When you say "Databricks Job", do you mean a custom SQL code you created in an stored procedure or function? Saying this because if you're really trying to run a real Databricks Job (lakeflow), it would be much better to establish another strategy different from that initial one. I mean, create firstly the Databricks Job and then, add type of tasks better for you scenario (https://docs.databricks.com/aws/en/jobs/configure-task#types-of-tasks )