Hi @sms101,

How are you doing today?

As per my understanding, It is correct that lineage tracking in Databricks works primarily at the table level, meaning when you reference a Delta table directly, the lineage is properly captured. However, when you use file paths as data sources, Databricks does not track lineage since it sees the source as just a file location, not a registered table. For better lineage visibility, consider registering your data sources as Delta tables before referencing them in queries. This will help ensure the lineage is consistently tracked. Additionally, maintaining consistent use of catalog tables instead of direct file paths is a recommended practice to preserve full lineage tracking across your workflow.

Please let me know if it works and have a good day.

Regards,

Brahma