DBR 14.1 - foreachBatch in Spark Connect Shared Clusters are not supported in Unity Catalog.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-09-2023 12:10 PM
I am getting this error on DBR 14.1
AnalysisException: [UC_COMMAND_NOT_SUPPORTED.WITHOUT_RECOMMENDATION] The command(s): foreachBatch in Spark Connect Shared Clusters are not supported in Unity Catalog.
This is the code:
wstream = df.writeStream.foreachBatch(foreach_batch_function).trigger(processingTime='1 seconds')
query=wstream.start()
The code works for DBR 13.3 LTS
is it a bug ?
