Basically all what is needed is to create api token in databricks and than use Jobs API as described here:
https://docs.databricks.com/dev-tools/api/latest/jobs.html
following endpoints are available:
POST https://<databricks-instance>/api/2.1/jobs/create - Create a new job.
GET https://<databricks-instance>/api/2.1/jobs/list - Retrieves a list of jobs.
GET https://<databricks-instance>/api/2.1/jobs/get - Retrieves the details for a single job.
POST https://<databricks-instance>/api/2.1/jobs/reset - Overwrites all the settings for a specific job. Use the Update endpoint to update job settings partially.
POST https://<databricks-instance>/api/2.1/jobs/update Add, update, or remove specific settings of an existing job. Use the Reset endpoint to overwrite all job settings.
POST https://<databricks-instance>/api/2.1/jobs/delete - Deletes a job.
POST https://<databricks-instance>/api/2.1/jobs/run-now - Run a job and return the run_id of the triggered run.
POST https://<databricks-instance>/api/2.1/jobs/runs/submit - Submit a one-time run.
GET https://<databricks-instance>/api/2.1/jobs/runs/list - List runs for a job
GET https://<databricks-instance>/api/2.1/jobs/runs/get Retrieve the metadata of a run.
GET https://<databricks-instance>/api/2.0/jobs/runs/export Export and retrieve a job run
POST https://<databricks-instance>/api/2.1/jobs/runs/cancel Cancel a job run
GET https://<databricks-instance>/api/2.1/jobs/runs/get-output - Get the output for a single run
POST https://<databricks-instance>/api/2.1/jobs/runs/delete - Deletes a non-active run
