- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-26-2024 10:50 AM
I have a notebook where I read multiple tables from delta lake (let say schema is db) and after that I did some sort of transformation (image enclosed) using all these tables lwith transformations like join,filter etc. After transformation and writing it to delta table, I am getting insight from databricks to use disk cache(image enclosed). In this scenario how can I use disk cache. I used
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-26-2024 11:54 AM
@anupam676 - could you please use set function instead of get
spark.conf.set("spark.databricks.io.cache.enabled","true")
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-27-2024 12:11 AM
Thank you @shan_chandra