We are investigating how to pass parameter from Databricks Job to DLT pipeline. Our process orchestrator is Azure Data Factory from where we trigger the Databricks Job using Jobs API. As part of the 'run-now' request, we would like to pass a parameter to the DLT pipeline task of our Job. Could you advise us how to achieve that?
Thanks!
