Parameterizing DLT Jobs

[leelee3000]

I have observed the use of advanced configuration and creating a map as a way to parameterize notebooks, but these appear to be cluster-wide settings. Is there a recommended best practice for directly passing parameters to notebooks running on a DLT cluster? Specifically, if I have multiple notebooks running simultaneously and want to pass unique parameters to each (e.g., {“name”:”alpha”} for notebook1 and {“name”:”beta”} for notebook2), how can I achieve this without impacting other notebooks on the same DLT cluster? #deltalivetables

[Kaniz]

Hi @leelee3000, In Databricks workflows, you can pass parameters to tasks that reference notebooks. For example, you can use the dbutils.jobs.taskValues.set function to register a parameter in the fir.... However, this method does not currently work for tasks that refer to Delta Live Tables (DLT) pipelin....

As of now, DLT tasks only support configuration parameters defined in the pipeline’s settings or Spa.... Therefore, you might not be able to pass unique parameters to each notebook running on the same DLT cluster without impacting other notebooks.

You might want to consider using the JSON configuration in DLT settings for more advanced options. This allows you to directly edit the JSON configuration in the workspace, which can be useful when d....