Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-25-2022 12:02 AM
yes. Spark is lazy evaluated meaning that it will execute code only on actions. display() is such an action, write too.
So Spark will first execute the query for the write (read, transform, write), and then do the same a second time for the display (read, transform, display). The left_anti join will return nothing as the data is added to the delta table.
if you would read/write from different tables, the df would not be empty (as the read table has not changed).