Databricks Community

Brahmareddy

Hey folks,Ever notice how a query that used to run super fast suddenly starts dragging? We’ve all been there. As data grows, those little inefficiencies in your SQL start showing up — and they show up hard. That’s where something cool comes in: using...

Brahmareddy

Hey Data Engineers!  How are you all doing? Hope you're having a great time working with data!Let me share a quick story. A while ago, my team had a problem—our data pipeline was taking 4+ hours to run. We tried different things, but nothing worked....

Brahmareddy

Data Engineering has come a long way. From the days of manual ETL scripts to the modern world of automated, AI-driven data pipelines, the evolution has been nothing short of fascinating. As a data engineer working across various platforms, I’ve seen ...

Brahmareddy · 02-17-2025

Hello Austin Databricks Community,Hope this article finds you well.Ever spent hours tweaking partitions, only to find out your queries are still slow? Yeah, we’ve all been there. But guess what? Liquid Clustering in Delta Lake 3.0 makes all that manu...

Brahmareddy · 02-06-2025

Hello Austin Databricks Community,I hope this message finds you well.Just wanted to share some new updates happening in Data + AI world right now.Nowadays, companies are making use of Data Mesh and Data Fabric to decentralize data ownership, making ...

Brahmareddy

Hi Dharanip,From what you’ve described, your logic looks good, but the issue comes from how Materialized Views work in Databricks. They only support incremental updates when the query is very simple and fully predictable (or "deterministic"). Since y...

Brahmareddy

Hi mydefaultlogin,How are you doing today?, As per my understanding, You're right—this happens because when you're running notebooks from your Git folder, Python knows exactly where your project root is and can easily find my_package. But when you de...

Brahmareddy

Hi Dharinip,How are you doing today? As per my understanding, it looks like your Materialized View is falling back to full recompute because the source data or query isn't eligible for incremental updates. Based on the message, there are a few blocke...

Brahmareddy

Hi hdu,How are you doing today?, As per my understanding, It sounds like you’re really close! That “No API found” error usually means either the wrong API endpoint is being used, or the cluster type doesn’t support ownership changes—for example, shar...

Brahmareddy

Hi Shivap,Totally get the confusion—it’s a common one! Even if you use an ADLS path when creating a catalog or schema, Unity Catalog still treats it as managed because Databricks controls the folder structure inside that path. So when you create tab...

Databricks Community

User Stats

User Activity

Use Query Patterns to Suggest Indexes Dynamically

Data Engineering: A Team Sport for Success!

The Future of Data Engineering: Smarter, Faster, and More Automated

Liquid Clustering: No More Partition Headaches!

Latest Trends in Data + AI for 2025

Re: Incrementalization issue in Materialized views

Re: Inconsistent PYTHONPATH, Git folders vs DAB

Re: Incrementalization issue in Materialized views

Re: Change cluster owner API call failed

Re: What's the recommended way of creating tables in Databricks with unity catalog (External/Managed