Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-12-2023 06:38 AM
Hi @erigaud Good day!
Whenever you are saving the data, you could pass the parquet.block.size config as an option:
Example:
spark.read.parquet("dbfs:/delta/delta-path/part-xxxx.snappy.parquet").write.mode("overwrite").option("parquet.block.size", 500).parquet("/tmp/vinay/parquet/blocksize1")