Databricks Community

RevanthV

Hi Team,Recently i came across a situation where I had to write a huge data and it took 6 hrs to complete...later when i checked the target data , I saw 20% of the total records written incorrectly or corrupted because the source data itself was corr...

RevanthV · 11-19-2025

I need to do some POC with spark 3.5.7 and 4.x and need some local setup with some sample Kafka source. The POC would read data from Kafka via streaming job and write to delta table and I would like to do this on spark-4.x ..Do you know of any quick ...

RevanthV · 09-13-2025

I have written data to a table using clusterByAuto set to trueBut the clustering keys are not selected automatically when i do a desc detail on the table.Screenshot belowWhy are clustering columns not being selected automatically?Repro steps:Create ...

RevanthV

Hey @K_Anudeep , thanks a lot for tagging me into the GitHub issue.. This is exactly what I want " validate and commit" feature and i se you have already raised a PR for the same with a new option called . I will try this out and check if it satisfie...

RevanthV · 11-19-2025

Hey Louis, Good Day , I have found all the above info on ChatGPT long time ago , but using any of the set up takes a lot of time and I needed something with can be quick and developer friendly.Last Friday I came across a post from @K_Anudeep on Sett...

RevanthV · 11-14-2025

Hey, I have tried this out on my laptop and it hardly takes 3 minutes to setup a cluster like env locally..This is really helpful.Great work @K_Anudeep 

RevanthV · 11-03-2025

Hey @K_Anudeep , Thanks for letting us know that this is a bug and is mitigated. I tried testing it today again as i was getting the same error last week but it doesn't occur now anymore

RevanthV · 09-13-2025

Thanks a lot @K_Anudeep , My table is still small and i guess that was the reason , have now written around 1million records and have been just running frequent scans on a column since last one hour and now i can see the same column selected as a cl...

Databricks Community

User Stats

User Activity

Data validation with df writes using append mode

POC on spark 4.x

Issue with Auto Liquid clustering

Re: Data validation with df writes using append mode

Re: POC on spark 4.x

Re: ⭐ Setup Spark with Hadoop Anywhere : A DBR aligned local Spark+HDFS+Hive stack on Docker⭐

Re: Broken s3 file paths in File Notifications for auto loader

Re: Issue with Auto Liquid clustering