Problem: Queries on our 100M+ record Iceberg tables were taking 45+ seconds.

Solution: Implemented lightweight zonemap indexing that tracks min/max values per file.

Quick Implementation

def apply_zonemap_pruning(table_path, predicate_value):
# Load zonemap index
zonemap = spark.read.parquet(f"{table_path}/_zonemaps")

# Filter files based on min/max values
relevant_files = zonemap.filter(
(zonemap.min_value <= predicate_value) &
(zonemap.max_value >= predicate_value)
).select("file_path").collect()

# Read only relevant files instead of a full table scan
return spark.read.parquet(*[f.file_path for f in relevant_files])
Results

Results

• Before: 42.3 seconds (scanning 1000 files)
• After: 5.4 seconds (scanning 12 files)
• Cost savings: 87% reduction in compute

Key insight: Most queries only need 1-2% of files. Zonemap helps identify them instantly.

We are currently adding Bloom filters for even better performance. Has anyone tried similar indexing strategies?

Would love to hear your approaches!