Creating a Databricks Asset Bundle with Sequential Pipelines and Workflow using YAML

smit_tw
Databricks Partner

Is it possible to create a repository with a Databricks asset bundle that includes the following pipelines?

  1. Test1 (Delta Live Table Pipeline)
  2. Test2 (Delta Live Table Pipeline)
  3. Test3 (Delta Live Table Pipeline)
  4. Workflow Job
  5. Workflow to execute the above pipelines in sequence (4 → 1 → 2 → 3).

Can you create 5 YAML files that accomplish the following:

  • Define and set up the pipelines.
  • Configure the workflow to run them in the specified sequence (4 → 1 → 2 → 3), ensuring each pipeline correctly references the ones it depends on?