Reading snappy.parquet
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-13-2025 02:58 AM
I stored a dataframe as delta in the catalog. It created multiple folders with snappy.parquet files. Is there a way to read these snappy.parquet files.
it reads with pandas but with spark it gives error "incompatible format"
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-13-2025 03:17 AM
Hello good day @Hritik_Moon
That incompatible format is expected as when you try to read in parquet because of presence of delta_log created with delta format which follows acid principals its like AnalysisException.
recommended would be read in delta format only
else: the alternative would be copy those .snappy.parquet files or file into a desired folder and read them seperately.
Let me share a medium article I found for this issue:
https://medium.com/%40ishanpradhan/how-to-read-a-snappy-parquet-file-in-databricks-696538cd0efc
Thank you.
I am waiting for the solution from other contributors as well. they can share their approach.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-16-2025 09:24 PM
Try to read the file as delta.
path/delta_file_name/
- parquet files
- delta_log/
since you are using spark, use this, spark.read.format("delta").load("path/delta_file_name").
Delta internally stores the data as parquet and delta log contains the metadata of transactions. You don't need to touch these files unless you are experimenting. 🙂
For more info, please go through this, https://docs.databricks.com/aws/en/delta/tutorial.
Hope this solved your issue.