Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-23-2023 05:17 AM
Yes, you can as GeneratedAlwaysAs is in delta schema, for example, write to the table (which is registered delta as the table).
df.write.format("delta").mode('append').saveAsTable('TableName')
some more examples are there https://docs.delta.io/latest/delta-batch.html there are a lot of ways to achieve that
DeltaTable.create(spark) \
.tableName("default.events") \
.addColumn("eventId", "BIGINT") \
.addColumn("data", "STRING") \
.addColumn("eventType", "STRING") \
.addColumn("eventTime", "TIMESTAMP") \
.addColumn("year", "INT", generatedAlwaysAs="YEAR(eventTime)") \
.addColumn("month", "INT", generatedAlwaysAs="MONTH(eventTime)") \
.addColumn("day", "INT", generatedAlwaysAs="DAY(eventTime)") \
.partitionedBy("eventType", "year", "month", "day") \
.execute()
My blog: https://databrickster.medium.com/