cancel
Showing results for 
Search instead for 
Did you mean: 
Community Articles
Dive into a collaborative space where members like YOU can exchange knowledge, tips, and best practices. Join the conversation today and unlock a wealth of collective wisdom to enhance your experience and drive success.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

TheOC
by Contributor
  • 358 Views
  • 17 replies
  • 61 kudos

Level Up Your Databricks Game - Episode 1: Widgets

Hi all,TheOC here with my first of (hopefully!) many blogs on the Databricks Community. I'm hoping in this series to share quick, practical tips to help you get the most out of Databricks. Today's topic is: Widgets.If you're anything like me, you've ...

TheOC_0-1756659655433.gif TheOC_1-1756659909004.gif ezgif-589e531c99345c.gif ezgif-5d5cf85fdf3734.gif
  • 358 Views
  • 17 replies
  • 61 kudos
Latest Reply
TheOC
Contributor
  • 61 kudos

hey @Rishabh_Tiwari,Thank you!Watch this space for the next instalment

  • 61 kudos
16 More Replies
Pat
by Esteemed Contributor
  • 177 Views
  • 4 replies
  • 11 kudos

[Blog] Building a Scalable Telco CDR Processing Pipeline with Databricks Delta Live Tables - Part 1

Hey everyone! I wanted to share what I'm working with daily in the Databricks ecosystem and how amazing it is that we can achieve everything within one platform!Just published a deep dive on building a Telco CDR Processing Pipeline using: Delta Live ...

  • 177 Views
  • 4 replies
  • 11 kudos
Latest Reply
BS_THE_ANALYST
Honored Contributor III
  • 11 kudos

@Pat here I was, cup of tea in hand, ready and eager to read the blog, only then did I discover I was redirected to another place that actually hosts the full blog .Personally, I think it'd be nice for it to be on here, especially if it's dedicated t...

  • 11 kudos
3 More Replies
Coffee77
by New Contributor II
  • 499 Views
  • 12 replies
  • 7 kudos

Resolved! 🚀 DataFrame Caching on Delta Tables - What if underlying data is updated?

Just published new video on Databricks Performance Series to try to clearly explain how DataFrame caching over Delta Tables behaves when updates on underlying table are performed. I came across this use case in my recent project and struggled a littl...

CAFE77_0-1756205776301.png
  • 499 Views
  • 12 replies
  • 7 kudos
Latest Reply
Coffee77
New Contributor II
  • 7 kudos

Source Code with samples available at https://github.com/CafeConData/Spark-Caching-on-Delta-Tables

  • 7 kudos
11 More Replies
Coffee77
by New Contributor II
  • 77 Views
  • 0 replies
  • 2 kudos

🚀 Spark Caching vs Databricks Disk Caching

As promised @BS_THE_ANALYST , in this new video and summarized in post, I try to explain what Spark Caching and Databricks Disk Caching are and how Caching strategy can be leveraged by making these cool features work together: Spark Caching vs Databr...

Coffee77_1-1756394420204.png Coffee77_0-1756394294274.jpeg
  • 77 Views
  • 0 replies
  • 2 kudos
sandy311
by New Contributor III
  • 80 Views
  • 0 replies
  • 1 kudos

Parallel Model Training & Data Pipelines on Databricks (ForEach Tasks+ Asset Bundles + Pydantic)

As companies double down on machine learning (ML), one thing is obvious: a single model can’t solve every problem. Different datasets, different timelines, and different requirements make managing multiple models pretty tricky. And if you’ve ever wor...

sandy311_0-1756395092044.png sandy311_1-1756395091510.png
Community Articles
ForEach
jobs
MLOPS
pydantic
python
  • 80 Views
  • 0 replies
  • 1 kudos
WiliamRosa
by New Contributor II
  • 586 Views
  • 3 replies
  • 7 kudos

Resolved! Data Quality with PySpark and Great Expectations on Databricks

Data governance is one of the most important pillars in any modern architecture. When building pipelines that process data at scale, ensuring data quality is not just a best practice—it is a critical necessity.Tools like Great Expectations (GX) were ...

  • 586 Views
  • 3 replies
  • 7 kudos
Latest Reply
BR_DatabricksAI
Contributor III
  • 7 kudos

@WiliamRosaWiliamRosa:  Thanks for sharing the link. I will explore. 

  • 7 kudos
2 More Replies
nayan_wylde
by Honored Contributor
  • 254 Views
  • 0 replies
  • 1 kudos

Tracking Query History and Optimizing Queries in Databricks

Optimizing queries in Databricks isn’t just about adding indexes or tweaking SQL syntax — it’s about visibility. You can’t improve what you can’t measure. Fortunately, Databricks provides rich telemetry around query history that you can use to analyz...

  • 254 Views
  • 0 replies
  • 1 kudos
Isi
by Honored Contributor II
  • 232 Views
  • 1 replies
  • 4 kudos

[Blog] Databricks Serverless vs Classic: Who Wins the Cost Sprint?

Hi everyone! I wanted to share with you a post I wrote on Medium a while ago — it’s still very useful if you want to understand how to properly calculate Databricks cluster costs and get a realistic view of the differences: Databricks Serverless vs C...

  • 232 Views
  • 1 replies
  • 4 kudos
Latest Reply
Coffee77
New Contributor II
  • 4 kudos

Really interesting topic I'll take a look when possible Always interested in improving performance and saving cloud costs. Thanks for sharing.

  • 4 kudos
Coffee77
by New Contributor II
  • 252 Views
  • 2 replies
  • 3 kudos

Resolved! 🚀 Boost Databricks Performance ✅ Lazy Evaluation and Spark DataFrame Caching ☕

Published new video in my recently created youtube channel about one of my favorite topics: performance.  So, here is a new video whose goal is to explain clearly what lazy evaluation is and how to use caching to boost performance.   https://studio.y...

CAFE77_4-1756040519167.png
  • 252 Views
  • 2 replies
  • 3 kudos
Latest Reply
Coffee77
New Contributor II
  • 3 kudos

Not sure if previous link works fine, here is the correct one https://youtu.be/pLGAr1VXQSQ?si=QaKzaNfzrNl_0Tv9

  • 3 kudos
1 More Replies
Coffee77
by New Contributor II
  • 315 Views
  • 4 replies
  • 3 kudos

Resolved! Introduction to Databricks for Beginners Video

Here is the first episode in ENGLISH VERSION of a list of simple videos on Introduction to Databricks for beginners:Introduction to Databricks for Beginners - Episode 1 It contains previous and basic concepts to master before moving forward with Data...

  • 315 Views
  • 4 replies
  • 3 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 3 kudos

Hi @Coffee77 ,Thanks for sharing with us. It looks promising. Can't wait for another episode

  • 3 kudos
3 More Replies
snehamore811
by New Contributor III
  • 950 Views
  • 4 replies
  • 7 kudos

Resolved! Databricks for RAG: Build, Run, Evaluate

What is RAG?RAG (Retrieval-Augmented Generation) on Databricks refers to building and running AI applications that combine:Retrieval systems (like vector databases or search over documents)Generative AI models (such as LLMs like GPT)within the Databr...

  • 950 Views
  • 4 replies
  • 7 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 7 kudos

Thanks for sharing @snehamore811 

  • 7 kudos
3 More Replies
WiliamRosa
by New Contributor II
  • 5883 Views
  • 5 replies
  • 8 kudos

Resolved! Databricks Machine Learning Professional Preparation

Recently I earned the Databricks Machine Learning Professional certification and wanted to share my study journey. Before the exam, I worked on a project as a data engineer alongside data scientists (ML models, LLMs, MLflow). That led me to build a p...

WiliamRosa_0-1755947321744.png
  • 5883 Views
  • 5 replies
  • 8 kudos
Latest Reply
WiliamRosa
New Contributor II
  • 8 kudos

Thanks a lot, my friend @BS_THE_ANALYST ! Really glad you found it useful . I’m sure when you dive into ML later this year, you’ll do awesome things with it. Appreciate the kind words about the project — means a lot! All the best to you too, and let’...

  • 8 kudos
4 More Replies
BS_THE_ANALYST
by Honored Contributor III
  • 543 Views
  • 7 replies
  • 20 kudos

Zero to Hero - Data Analysis Certification

Hey everyone, Today I passed my Data Analyst Associate exam! 拾I'd like to say thanks to the Databricks community. The questions that people ask and interactions I've been a part of have been invaluable. Starting tomorrow, I'll be moving onto my data ...

  • 543 Views
  • 7 replies
  • 20 kudos
Latest Reply
TheOC
Contributor
  • 20 kudos

Congratulations on the Certifications @BS_THE_ANALYST  / @Pilsner .Excited to get the cert myself and race for the Data Engineering cert

  • 20 kudos
6 More Replies
Coffee77
by New Contributor II
  • 510 Views
  • 4 replies
  • 3 kudos

Resolved! Introduction to Databricks 🇪🇸

Here is the first episode of a serie of simple videos on Introduction to Databricks for beginners :https://youtu.be/kvglz79Ob-M?si=KnyCH74_HQ8jiO7SIt contains previous and basic concepts to master before moving forward with Databricks.

  • 510 Views
  • 4 replies
  • 3 kudos
Latest Reply
Coffee77
New Contributor II
  • 3 kudos

English version is ready : INTRODUCTION to DATABRICKS in English - Episode 1 ‌ 

  • 3 kudos
3 More Replies
TejeshS
by Contributor
  • 1165 Views
  • 3 replies
  • 8 kudos

Attribute-Based Access Control (ABAC) in Databricks Unity Catalog

What Is ABAC and Why Does It Matter?Attribute-Based Access Control (ABAC) is a data governance model now available in Databricks, designed to offer fine-grained, dynamic, and scalable access control for data, AI assets, and files managed through Data...

TejeshS_0-1753447608855.png TejeshS_1-1753447608858.png TejeshS_2-1753447608859.png TejeshS_3-1753447608861.jpeg
  • 1165 Views
  • 3 replies
  • 8 kudos
Latest Reply
Prashant2
New Contributor II
  • 8 kudos

You have mentioned 4 types of ABAC Policies. Since ABAC is now in Beta, we are trying to evaluate it for use cases for a client.We are looking for the GRANT and DENY types on tables based on Tags, however we don't see that even after enabling the fea...

  • 8 kudos
2 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels