You can provide parameters in the configuration section of DLT pipeline and access it in your code using spark.conf.get(<parameter_name>).

Parameterize DLT pipelines 

Thank you @AmanSehgal ,

I have done that and was able to execute the pipeline successfully. Bu t I need to change the parameter value at run time, so that the same pipeline can be used for multiple sources.

Can we pass parameters from Job to DLT Pipeline ?

I tried but it doesn't seem to be working. I tried passing job parameters to cluster configuration as {{job.paramters.name}} but it didn't pickup.

You might have to put a wrapper logic around your pipeline for now to parameterize it. For instance, write some text into a file in mounted location and then read it in the DLT pipeline for your conditional logic.

Hi @AmanSehgal , Yes, thats true, I had tried different options here. Thank you so much for taking it time to investigate this.