sandeepmankikar
Databricks Partner

To manage Databricks jobs within a DevOps pipeline, start by exporting the job configuration as a JSON file from the Databricks workspace. Parameterize this JSON by replacing environment-specific values with placeholders. Integrate the parameterized JSON into your DevOps repository and use CI/CD tools to replace these placeholders with actual values during pipeline execution. Create the job using the Databricks CLI command databricks jobs create --json-file path_to_your_json_file.json. Additionally, set up a separate pipeline to delete jobs using the command databricks jobs delete --job-id .