DLT Issue

[Databrickissue]

I have one DLT pipeline in Databricks. When I schedule the pipeline, the data is not showing. However, when I run the pipeline manually, the data is displayed properly

[Yogasathyandrun]

A few details would help narrow this down.

When the scheduled run executes:

1. Does the pipeline update show Succeeded or Failed?

2. In the pipeline Event Log, do you see rows being processed/written?

3. Is your manual run a normal update or a Full Refresh?

4. Is the schedule configured directly on the DLT pipeline or triggered through a Workflow/Job?

Common causes are:

• No new data available at the scheduled time (the update succeeds but processes 0 rows).

• Incremental processing vs Full Refresh (manual full refresh reprocesses historical data, scheduled updates only process new data).

• Different configuration/parameters between scheduled and manual runs.

• Permission differences if the pipeline is being triggered through another service or workflow.

The first thing I'd check is the Event Log for a scheduled run and compare it to a successful manual run. If both succeed but the scheduled run shows 0 records processed, the issue is usually related to source data availability or incremental processing rather than the pipeline logic itself.