Hello,I am using databricks free edition, is there a way to turn off IO caching.I am trying to learn optimization and cant see any difference in query run time with caching enabled.
I stored a dataframe as delta in the catalog. It created multiple folders with snappy.parquet files. Is there a way to read these snappy.parquet files.it reads with pandas but with spark it gives error "incompatible format"
Hello, I have created data frame on csv file when I try to write it as:df_op_clean.write.format("delta").save("/Volumes/optimisation/trial")I get this error :Cannot access the UC Volume path from this location. Path was /Volumes/optimisation/trial/_d...
Thanks for this. I have completed "Data Ingestion with Lakeflow connect" but couldn'd find other topics.Could you please help to provide the exact resource for learning.
