DLT behaving differently when used with python syntax vs when used with sql syntax to read CDF

Puspak
New Contributor III

‎01-30-2025 01:00 AM

I was trying t read CDF data of a table as a DLT materialized view.

It works fine with sql syntax reading all the columns of the source table along with the 3 CDF columns : _change_type,_commit_timestamp,_commit_version:

@dlt.table()
def change_table():
    df_change = spark.sql("SELECT * FROM table_changes(<source_table_name>,1)")
    return(df_change)
 
But when I try the same with python it just reads the columns of the source table leaving out the CDF columns : _change_type,_commit_timestamp,_commit_version:
@dlt.table()
def change_table():
    df_change = spark.read.option('readChangeFeed','True').option('startingVersion',1).table(<source_table_name>)
    return(df_change)
0 Kudos