Databricks Community

NemesisMF

Is it possible to obtain somehow if a DLT pipeline run is running in Full Refresh or incremental mode from within a notebook running in the pipeline?I looked into the pipeline configuration variables but could not find anything.It would be benefitial...

NemesisMF

We found a solution where we do not need to determine the refresh mode anymore. But I still do not know how to get the current refresh mode of the current pipeline run from within a notebook that is running in the pipeline. This may would still be be...

NemesisMF

Thanks for the quick answer. Where did you get theget_current_pipeline_run_config()from? I used spark.conf.getAll which apperently does not have the refresh mode info.StartUpdatecomes from the databricks.sdk.service.pipelines?

Databricks Community

User Stats

User Activity

Obtain refresh mode from within Delta Live Table pipeline run

Re: Obtain refresh mode from within Delta Live Table pipeline run

Re: Obtain refresh mode from within Delta Live Table pipeline run