Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-08-2022 12:00 AM
Hi, @Vincent Doe ,
Updates are available in Delta tables, but under the hood you are updating parquet files, it means that each update needs to find the file where records are stored, then re-write the file to new version, and make new file current version.
In your case maybe you should try something like this:
spark.sql("""
select
col1,
col2,
col3,
case
when id = '123' then '2022-02-16'
when id = '124' then '2022-02-17'
end as create_date
...
from example_view
""") \
.write \
.mode('overwrite') \
.saveAsTable('example_table')