Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-12-2020 07:22 PM
This is very helpful ... Thanks for the information ... Just to add more info to it if somebody wants to disable it at Cluster level for Spark 2.4.5. they can edit the Spark Cluster -) Advanced Options and add above but you need to use <variable> <value> like below :
parquet.enable.summary-metadata false
If you want to add it in databricks notebook you can do like this:
spark.conf.set("parquet.enable.summary-metadata", "false")