Databricks SDP (Spark declarative Pipelins) Overwrite table
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sunday
1) Consider I have orders folder which orders_1.csv file and it is loaded to orders tables
orders/ ============> load to order table
orders_1.csv
2)
next day new file (orders_2.csv) arrived
orders/ =========> take only 2nd file -----> overwrite the order table
orders_1.csv
orders_2.csv
Here streaming table only appending the data to orders table. Materialized view is considering both files.
My requirement is overwrite the orders table.
Can you help me how to do that ??
