topic Re: The operation CHANGE DATA FEED is not allowed on Streaming Tables. in Data Engineering

The operation CHANGE DATA FEED is not allowed on Streaming Tables.

erigaud — Mon, 27 Nov 2023 15:26:21 GMT

Hello everyone,

I have a workflow that starts by reading the CDF data for a change data feed.

The syntax is exactly the following :

(

spark.readStream

.format("delta")

.option("readChangeFeed", "true")

.option("startingVersion", 10)

.table("my.streaming.table")

.display()

)

I get the following error :

[STREAMING_TABLE_OPERATION_NOT_ALLOWED.UNSUPPORTED_OPERATION] The operation CHANGE DATA FEED is not allowed: The operation is not supported on Streaming Tables.

I made sure the version exists in the history of the table, and the CDF is enabled for my table.

A couple of weeks ago the code used to run without issue, did something change ? Thank you

Re: The operation CHANGE DATA FEED is not allowed on Streaming Tables.

AG2 — Mon, 27 Nov 2023 15:34:25 GMT

To facilitate this, there are certain specific operations that are allowed on streaming tables. If you encounter this error, it's likely that the operation you're attempting to use is not one of the supported ones. You will need to review the documentation for streaming tables to determine which operations are allowed and how to properly use them.

Re: The operation CHANGE DATA FEED is not allowed on Streaming Tables.

erigaud — Mon, 27 Nov 2023 16:00:03 GMT

The operation I'm attempting is supported, it's literally the same thing as what is showed in the documentation (https://docs.databricks.com/en/delta/delta-change-data-feed.html)

Moreover as I said the code used to work just a couple of weeks ago.

Re: The operation CHANGE DATA FEED is not allowed on Streaming Tables.

afk — Tue, 19 Dec 2023 11:51:27 GMT

Hi, this seems to be related to the issue I've been getting around the same time here:
Change data feed from target tables of APPLY CHANG... - Databricks - 54436

Would be great to get an explanation for the sudden change in behaviour.