Databricks

erigaud · ‎11-27-2023

Hello everyone,

I have a workflow that starts by reading the CDF data for a change data feed.

The syntax is exactly the following :

(

spark.readStream

.format("delta")

.option("readChangeFeed", "true")

.option("startingVersion", 10)

.table("my.streaming.table")

.display()

)

I get the following error :

[STREAMING_TABLE_OPERATION_NOT_ALLOWED.UNSUPPORTED_OPERATION] The operation CHANGE DATA FEED is not allowed: The operation is not supported on Streaming Tables.

I made sure the version exists in the history of the table, and the CDF is enabled for my table.

A couple of weeks ago the code used to run without issue, did something change ? Thank you

AG2 · ‎11-27-2023

To facilitate this, there are certain specific operations that are allowed on streaming tables. If you encounter this error, it's likely that the operation you're attempting to use is not one of the supported ones. You will need to review the documentation for streaming tables to determine which operations are allowed and how to properly use them.

erigaud · ‎11-27-2023

The operation I'm attempting is supported, it's literally the same thing as what is showed in the documentation (https://docs.databricks.com/en/delta/delta-change-data-feed.html)

Moreover as I said the code used to work just a couple of weeks ago.

Kaniz · ‎11-27-2023

Hi @erigaud , Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!