Incremental ingestion of Snowflake data with Delta Live Table (CDC)

Khalil
Contributor

Hello,

I have some data which are lying into Snowflake, so I want to apply CDC on them using delta live table but I am having some issues.

Here is what I am trying to do:

 

 

@dlt.view()
def table1():
   return spark.read.format("snowflake").options(**options).option('query', query).load()

def.create_streaming_table(target)
dlt.apply_changes(
source = 'table1'
target = 'target'
....
)

 

 

The same code run well if I am reading a delta table but if its snowflake am having the following error

'org.apache.spark.sql.AalysisException: Source data for the APPLY CHANGES target 'XXXXX' must be a streaming query'

Is there a solution or a workaround you can help me with?