- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-17-2025 08:51 AM - edited 11-17-2025 08:52 AM
Pipeline cost
ADF charges per activity run + Data Integration Units (DIUs) used for copy/mapping activities, plus Integration Runtime time.
LakeFlow charges per compute usage, not per-step. Fewer moving pricing knobs, but your cost is dominated by:
cluster size/type
runtime (how long the job runs)
how often you trigger it
You will pay also for storage.
For many migrations, you replace a large ADF pipeline full of small activities with one or a few LakeFlow jobs, so you trade lots of per-activity charges for a simpler, cluster-based cost model.
Job reuse
Within a single LakeFlow job run you can reuse the same compute across multiple notebook tasks – if you configure it that way. Each job run itself is independent.
A LakeFlow job is a DAG of tasks (notebooks, pipelines, Python scripts, etc.).
Each task has a compute configuration:
shared job cluster (recommended), or
its own cluster, or
serverless, or
an existing interactive cluster.
If you attach all notebook activities in that job to the same job cluster / serverless definition, then in one run:
the cluster is started once,
all tasks run on that same cluster,
the cluster is then terminated based on your settings.
It does not automatically reuse compute across different job runs or across different jobs, unless you deliberately target a long-running interactive cluster.
For an ADF pipeline with N Databricks notebook activities, the usual pattern is:
create one LakeFlow job with N notebook tasks,
attach them all to one shared job cluster → this typically reduces cost vs many independent jobs.