Hi @Mario_D 

Great question. I've run into this exact issue before in my own projects! When lineage suddenly disappears, it's almost never an API restriction. Instead, it's usually one of three things happening under the hood in Unity Catalog:

• Lost Permissions (Most Common): Unity Catalog hides lineage for security if your user or Service Principal lost BROWSE or SELECT access to the upstream tables between Week 1 and Week 2. Check if you can still see those upstream tables in the Catalog Explorer!

• Table Re-creation: If someone (or a job) dropped and recreated the upstream table instead of just updating it, the historical lineage link breaks.

• Pipeline Code Changes: Lineage is built dynamically from Spark execution plans. If someone changed the ETL notebook/job and it no longer actually pulls from that upstream column, the lineage will update to reflect that.

One quick catch on your code: I noticed a typo in your snippet! You wrote lineage['ustream_cols'] instead of upstream_cols (missing the 'p'). If that is exactly how it is in your live script, it will definitely fail to pull the data!

Hope this points you in the right direction!