Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-04-2025 01:20 PM - edited 09-04-2025 01:21 PM
@ck7007 brainstormed some solution approaches ., do you have some test data to test these hands on
Approach Throughput Query Speed Complexity Notes
| Partition-level zonemaps | High | Medium | Low | Scales with micro-batches; prune at partition/file level |
| File consolidation / Optimize | Medium | High | Medium | Reduces metadata churn; needs tuning for latency vs file size |
| Deferred global index | High | Medium-High | Medium | Preserves streaming throughput; query may hit unoptimized files |
| Bloom filters / secondary index | High | High | Medium | Low false positives; good for selective queries |
| Delta predictive optimization | High | Medium-High | Low | Fully managed; minimal operational overhead |
