Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-31-2025 11:26 PM
Sorry, I am not going through the entire schema and code, but in general the error "java.lang.ArithmeticException: long overflow" typically occurs when a calculation exceeds the range that can be represented by a long data type in Java. This issue can arise in various scenarios, including data type conversions, arithmetic operations, or when handling large datasets.
You could try the below conf.
spark.sql.parquet.int96RebaseModeInWrite to CORRECTED