DBR 14.1 - foreachBatch in Spark Connect Shared Clusters are not supported in Unity Catalog.

[samur]

I am getting this error on DBR 14.1

AnalysisException: [UC_COMMAND_NOT_SUPPORTED.WITHOUT_RECOMMENDATION] The command(s): foreachBatch in Spark Connect Shared Clusters are not supported in Unity Catalog.

This is the code:

wstream = df.writeStream.foreachBatch(foreach_batch_function).trigger(processingTime='1 seconds')

query=wstream.start()

The code works for DBR 13.3 LTS

is it a bug ?

[Anonymous]

Hi - Can you provide some more information on this?

"This limitation is specific to the version you’re using."... was this removed in 14.1 but worked in older DBR's? Is it going to be returning? It's really getting difficult to keep up with these run time changes.