Databricks Community

RevanthV

I need to do some POC with spark 3.5.7 and 4.x and need some local setup with some sample Kafka source. The POC would read data from Kafka via streaming job and write to delta table and I would like to do this on spark-4.x ..Do you know of any quick ...

RevanthV · 09-13-2025

I have written data to a table using clusterByAuto set to trueBut the clustering keys are not selected automatically when i do a desc detail on the table.Screenshot belowWhy are clustering columns not being selected automatically?Repro steps:Create ...

RevanthV

Hey Louis, Good Day , I have found all the above info on ChatGPT long time ago , but using any of the set up takes a lot of time and I needed something with can be quick and developer friendly.Last Friday I came across a post from @K_Anudeep on Sett...

RevanthV

Hey, I have tried this out on my laptop and it hardly takes 3 minutes to setup a cluster like env locally..This is really helpful.Great work @K_Anudeep 

RevanthV · 11-03-2025

Hey @K_Anudeep , Thanks for letting us know that this is a bug and is mitigated. I tried testing it today again as i was getting the same error last week but it doesn't occur now anymore

RevanthV · 09-13-2025

Thanks a lot @K_Anudeep , My table is still small and i guess that was the reason , have now written around 1million records and have been just running frequent scans on a column since last one hour and now i can see the same column selected as a cl...

Databricks Community

User Stats

User Activity

POC on spark 4.x

Issue with Auto Liquid clustering

Re: POC on spark 4.x

Re: ⭐ Setup Spark with Hadoop Anywhere : A DBR aligned local Spark+HDFS+Hive stack on Docker⭐

Re: Broken s3 file paths in File Notifications for auto loader

Re: Issue with Auto Liquid clustering