Hi there !!! I would like to find the most suitable orchestration process to promote data between medallion layers I need to solve the following key architectural decision for scaling my multi-tenant data lake in Databricks.

My setup:

• Independent medallion architecture per client (Landing → Bronze → Silver → Gold per client)
• Identical schema across all clients (same data model)
• Multiple tables per layer (each with specific transformations)

What would be the best approach in Databricks to orchestrate the data promotion between layers ?

• Independent pipelines per client for all tables
• Independent pipelines per client and table
• Independent pipelines per table for all clients

Thanks in advance.