🌟 Community Pulse: Your Weekly Roundup! October 2...

Sujitha · ‎11-01-2025

It’s been an exciting week across the Databricks Community — with new events, product announcements, and plenty of insightful technical discussions. Here’s a quick roundup of what’s been happening 👇

💬 Top Contributors of the Week

Kudos to our most active members for helping peers, sharing insights, and keeping the discussions vibrant!

🎉 Highlights from the Community

Regional Events:
- Portland Data + AI Meetup — Holiday Event
- Databricks Pittsburgh User Group — November 12
- Databricks Pune User Group Meetup — November 15 at EPAM Systems
- Databricks DevConnect | Washington D.C.
- New: Databricks Interactive Architecture Design Workshops
Product Announcements:
- Databricks Data Intelligence Platform Cheat Sheet
- Backfill Runs in Lakeflow Jobs for Higher Quality Downstream Data
- Find Sensitive Data at Scale with Data Classification in Unity Catalog
- Solution Accelerator #6 – Adverse Drug Event Detection
New Technical Blogs:
- Tutorial: Using Databricks Apps as a Lambda with Zerobus to write to Delta
- Container Service: Zerobus-Station
- Tutorial: Streamline Data Ingestion with Zerobus Ingest: RabbitMQ to the Lakehouse
- Zerobus Ingest in Action: How to Stream Event Data Directly into Your Lakehouse
Articles & Best Practices:
- Benchmarking: Free Edition
- Best Practices for Structuring Databricks Workspaces for CI/CD and ML Workflows
Hot Discussions:
- Agentic AI Workload Analyzer
- Tips for Streamlining Spark Job Development and Debugging in Databricks
- When to Use (and When Not to Use) Liquid Clustering
- Migrating from on-premises HDFS to Unity Catalog

🧩 Top Resolved Posts

Here are some of the key community questions that found solutions this week:

Intermittent 400 Error with Power BI Desktop - ODBC Connection to SQL Warehouse
Spark JDBC Write Fails for Record Not Present - PK error
Pass parameters between jobs
Notebook Session Has Crashed
Can a Databricks Workspace be renamed after creation?

💡 Trending Troubleshooting Topics

[NUMERIC_VALUE_OUT_OF_RANGE.WITHOUT_SUGGESTION] rounding error
Quota limit exhausted during SQL Server Connector ingestion
SQL Stored Procedures – Notebook CREATE query
Data ingestion from APIs (Shopify, Meta Ads, Google Ads, etc.)
Delta Share visibility issues

Thanks to everyone who contributed, shared knowledge, or joined an event this week!
Keep exploring, keep sharing, and stay tuned for more updates from the Databricks Community. 🚀

szymon_dybczak · ‎11-03-2025

Thank you @Sujitha ! Kudos to our members for their contribution 🙂

BS_THE_ANALYST · ‎11-03-2025

@szymon_dybczak beat me to it 🤣👆. Thanks @Sujitha 🙏.

We're getting closer to Christmas 👀🎄 ..

All the best,
BS

harlay · 3 weeks ago

Thanks for putting this weekly roundup together! It is always great to catch up on what the community is focusing on.

This week, I have been spending most of my time building data ingestion pipelines for the e-commerce side of things. We are currently moving a massive amount of raw order and customer data into Databricks using Delta Live Tables so we can run better predictive forecasting models.

It has been an interesting challenge. I have noticed that a lot of brands initially lean heavily on shopify development services to get their storefronts running and handle basic built-in reporting. But once the order volume scales up and you need to combine that transactional data with external marketing sources, moving everything into a lakehouse architecture like Databricks becomes the only way to really make sense of it all.

Has anyone else here been working on pipelines specifically for high-volume retail or e-commerce? I would love to hear how you are handling real-time inventory syncs without hitting rate limits on the source APIs.

Thanks again for the updates!