cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
PriyaAnanthram
by • Contributor III
  • 2253 Views
  • 6 replies
  • 0 kudos

Resolved! change data feed on delta live tables

I have a delta live table where I am reading cdc data and merging this data in silver using apply changes. In silver can I find out what all data has changed since the last run similar to change data feed table_changes?

Data Engineering
Reply
  • 2253 Views
  • 6 replies
  • 0 kudos
View Replies
Latest Reply
PriyaAnanthram
Contributor III
  • 0 kudos

I also have a requirment where i write to a live table (materialized view) and have cdf enabled i want to see the changes but here to i see overwrites happening after dlt pipeline runs

  • 0 kudos
5 More Replies
nimble
by • New Contributor
  • 1302 Views
  • 2 replies
  • 0 kudos

How can I run a streaming query on a new table with tbl property: change data feed enabled?

In Databricks on AWS, I am trying to run a streaming query (trigger=Once) with delta.enableChangeDataFeed=true in the table definition as instructed, but this always fails with :ERROR: Some streams terminated before this command could finish!   com.d...

Data Engineering
Reply
  • 1302 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
swethaNandan
New Contributor III
  • 0 kudos

Hi @daniel e​ Can you try running the select command on table changes from 0th version and see if you get output?SELECT * FROM table_changes('tableName', 0)Also, Please share the streaming query that you are running.

  • 0 kudos
1 More Replies
yatharthmahesh
by • New Contributor III
  • 1363 Views
  • 4 replies
  • 6 kudos

ENABLE CHANGE DATA FEED FOR EXISTING DELTA-TABLE

I have a delta table already created, now I want to enable the change data feed. I read that I have to set delta.enableChangeDataFeed property to true. But however, this cannot be done using the Scala API. I tried using this but it didn't work. I am ...

Data Engineering
Reply
  • 1363 Views
  • 4 replies
  • 6 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 6 kudos

Hi @Yatharth Maheshwari​, We haven’t heard from you on the last response from @Jose Gonzalez​, and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please do share that with the community as it can be helpf...

  • 6 kudos
3 More Replies
Ian
by • New Contributor III
  • 2879 Views
  • 6 replies
  • 0 kudos

Resolved! Databricks-Connect and Change Data Feed query error

I have installed Databricks-Connect (9.1 LTS). I am able to send queries to the cluster. However, when the query includes a call to the 'table_changes' function that is a part of Change Data Feed, I get the following error:AnalysisException("could ...

Data Engineering
Reply
  • 2879 Views
  • 6 replies
  • 0 kudos
View Replies
Latest Reply
Ian
New Contributor III
  • 0 kudos

Hi @Kaniz Fatma​ , the table_changes function is an internal Databricks function used in Change Data Feed (CDF).Please refer to the article below. It discusses the table_changes function.https://docs.databricks.com/delta/delta-change-data-feed.html

  • 0 kudos
5 More Replies
Labels