cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
kanikvijay9
New Contributor III
since ‎02-17-2025
Monday

User Stats

  • 4 Posts
  • 1 Solutions
  • 0 Kudos given
  • 9 Kudos received

User Activity

Optimizing Delta Table Writes for Massive Datasets in Databricks

3 weeks ago
Problem StatementIn one of my recent projects, I faced a significant challenge: Writing a huge dataset of 11,582,763,212 rows and 2,068 columns to a Databricks managed Delta table.The initial write operation took 22.4 hours using the following setup:...

Performance Issues with Writing Large DataFrames to Managed Tables in Databricks (3.5B+ Rows)

08-12-2025
Hi Community,I'm working on a large-scale data processing job in Databricks and facing performance and stability issues during the write operations. Here's a detailed breakdown of my use case and environment:Use Case Overview:Primary Data Frames:Firs...