is there a way to write streaming data to delta and create a table in spark catalog at the same time?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-25-2021 12:58 PM
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-25-2021 03:03 PM
Yes, in your write stream you can save it as a table in the delta format without a problem. In DBR 8, the default table format is delta.
See this code, please note that the "..." is supplied to show that additional options may be required:
df.writeStream.format("delta")...saveAsTable("<tablename>")

