Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-13-2025 06:26 AM
Now I understand how it's automatically configured in our cluster along with the rationale behind this off-heap memory approach.
However, I have some concerns about this configuration:
- General applicability: Most jobs don't actually require 70% off-heap memory allocation
- Industry recommendations: Leading LLM models (Claude, GPT, DeepSeek AI) don't recommend such high off-heap memory usage. Suggesting very very less % that is from the executor memory.
- Lack of benchmarks: I haven't found any test results or benchmarks supporting this configuration for caching or other workloads, even for GC optimization
- Cost implications: While this might help in some edge cases, it doesn't seem beneficial for general use cases and could be significantly increasing our costs
Could you please share any benchmark data or test results you have for this specific job configuration? This would help us better understand the performance benefits versus the cost impact.
Best regards,
Sowanth