cancel
Showing results for 
Search instead for 
Did you mean: 
Community Articles
Dive into a collaborative space where members like YOU can exchange knowledge, tips, and best practices. Join the conversation today and unlock a wealth of collective wisdom to enhance your experience and drive success.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

WiliamRosa
by New Contributor III
  • 434 Views
  • 1 replies
  • 0 kudos

Resolved! Automating Notebook Documentation in Databricks with LLMs

In one of my projects, I needed to generate structured documentation for an entire directory of Databricks notebooks.This solution uses the Databricks Workspace API together with a Serving Endpoint (LLM) to automatically create HTML documentation for...

  • 434 Views
  • 1 replies
  • 0 kudos
Latest Reply
WiliamRosa
New Contributor III
  • 0 kudos

Suggestions are always welcome — I hope this helps anyone looking to automate notebook documentation in Databricks.

  • 0 kudos
BS_THE_ANALYST
by Esteemed Contributor
  • 283 Views
  • 2 replies
  • 7 kudos

Data Security at the level of columns or rows or Data masking

Hi everyone, I'm currently going through the Data Analyst learning path. I've just learned about Dynamic Views and I wanted to share the article on them: https://docs.databricks.com/aws/en/views/dynamic#before-you-begin There are some limitations on ...

BS_THE_ANALYST_0-1755383776603.png
  • 283 Views
  • 2 replies
  • 7 kudos
Latest Reply
BigRoux
Databricks Employee
  • 7 kudos

@BS_THE_ANALYST  Cool stuff, right!  Have read about Attribute-based Access Control (ABAC) yet? Check it out: https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/ Let me know what you think. Cheers, Louis.

  • 7 kudos
1 More Replies
ilir_nuredini
by Honored Contributor
  • 174 Views
  • 0 replies
  • 2 kudos

Databricks Asset Bundles with Python!

Databricks Asset Bundles now support Python!If you’re a Python fan, you can define jobs and pipelines in Python (or keep YAML) with DABs. You can create jobs from simple metadata, modify them during deployment with mutators, and convert existing jobs...

  • 174 Views
  • 0 replies
  • 2 kudos
devipriya
by New Contributor III
  • 607 Views
  • 2 replies
  • 3 kudos

Getting Databricks Generative AI Engineer Associate (and What I Learned)

I just earned my Databricks Certified Generative AI Engineer Associate Certification, and in this post, I’m sharing the key tips, resources, and personal insights that helped me succeed. My certiticate from DatabricksNavigation:PreludeAbout the certR...

devipriya_0-1755492307055.png devipriya_1-1755492307055.png devipriya_2-1755492307056.png devipriya_3-1755492307056.png
  • 607 Views
  • 2 replies
  • 3 kudos
Latest Reply
BR_DatabricksAI
Contributor III
  • 3 kudos

Thank you @devipriya for providing the detailed information on certifications. 

  • 3 kudos
1 More Replies
RahulGupta
by New Contributor III
  • 329 Views
  • 0 replies
  • 1 kudos

Lakehouse vs. Lakehouse Federation - Bridging the Next Evolution in Data Platforms

Over the past few years, the Lakehouse architecture has become the gold standard for managing modern data workloads. By combining the low-cost storage of data lakes with the reliability and performance of data warehouses, Lakehouses have redefined ho...

RahulGupta_0-1755362435722.png
Community Articles
Databricks Lakehouse DataEngineering LakehouseFederation
  • 329 Views
  • 0 replies
  • 1 kudos
devipriya
by New Contributor III
  • 385 Views
  • 0 replies
  • 2 kudos

Understanding Modern Databricks Warehousing for the AI era: A Beginner’s Guide

 IntroductionIn the current Gen AI buzz, most conversations focus on RAG for unstructured documents. But there’s another equally important challenge — making sense of structured data at scale.This is where tools like Databricks Genie step in, enablin...

devipriya_0-1755110417311.png devipriya_1-1755110416034.png devipriya_2-1755110416212.png devipriya_3-1755110416450.png
  • 385 Views
  • 0 replies
  • 2 kudos
DILIPKHANDELWAL
by New Contributor
  • 285 Views
  • 0 replies
  • 0 kudos

Apache Spark 4.0 — Big Data Engineering!

The latest Spark 4.0 release delivers powerful enhancements across SQL, Python, streaming, and connectivity — all aimed at making big data workloads more efficient, reliable, and developer-friendly.With Databricks Runtime 17.0, these capabilities are...

  • 285 Views
  • 0 replies
  • 0 kudos
devipriya
by New Contributor III
  • 2529 Views
  • 0 replies
  • 1 kudos

Pipelines to Prompts: Getting started with Databricks and AWS

NAVIGATION:Why Data EngineeringThe Role of Data Engineering in GenAIWhat is Databricks? Unifying Data and AI on One PlatformDatabricks on AWS: A Full-Stack Platform for GenAIHands-On ExerciseFuture-Proofing: Why Data + AI Skills Matter Now More Than ...

Luluspace34_0-1754719688416.png Luluspace34_1-1754719687902.png Luluspace34_2-1754719687532.png Luluspace34_3-1754719688664.png
  • 2529 Views
  • 0 replies
  • 1 kudos
RahulGupta
by New Contributor III
  • 1284 Views
  • 1 replies
  • 3 kudos

Understanding Liquid Clustering in Databricks - The Next Evolution in Data Optimisation

In the world of big data, organising data smartly is just as important as collecting it. When working with large datasets in Databricks using Delta Lake, how your data is stored and ordered can greatly impact performance, especially for queries. Trad...

  • 1284 Views
  • 1 replies
  • 3 kudos
Latest Reply
BigRoux
Databricks Employee
  • 3 kudos

Great post, Rahul! You’ve nailed the key trade-offs perfectly. The Appeal: LC is “set it and forget it” data management—no more manual OPTIMIZE jobs or performance firefighting. The Reality Check: Single-column clustering works great for high-cardina...

  • 3 kudos
Charansai
by New Contributor III
  • 745 Views
  • 2 replies
  • 0 kudos

Recommendations for Designing Cluster Policies Across Dev/QA/Prod Environments for DE and DA teams

Hi Community,We are working on implementing Databricks cluster policies across our organization and are seeking advice on best practices to enforce governance, security, and cost control across different environments.We have two main teams using Data...

  • 745 Views
  • 2 replies
  • 0 kudos
Latest Reply
Charansai
New Contributor III
  • 0 kudos

I just want to confirm one more thing here is that me as admin managing the cluster creation and no user will have access to create them me know how the cluster policies help me in this perspective.

  • 0 kudos
1 More Replies
RiyazAliM
by Honored Contributor
  • 1684 Views
  • 4 replies
  • 7 kudos

The Open Source DLT Meta Framework

DLT Meta is an open-source framework developed by Databricks Labs that enables the automation of bronze and silver data pipelines through metadata configuration rather than manual code development.At its core, the framework uses a Dataflowspec - a JS...

aayrm5_1-1753154791734.png aayrm5_3-1753155396659.png aayrm5_4-1753155477157.png aayrm5_2-1753155019763.png
  • 1684 Views
  • 4 replies
  • 7 kudos
Latest Reply
sridharplv
Valued Contributor II
  • 7 kudos

Great Article Riyaz. keep Sharing more knowledge

  • 7 kudos
3 More Replies
Ajay-Pandey
by Esteemed Contributor III
  • 4035 Views
  • 3 replies
  • 5 kudos

Cross-filtering for AI/BI dashboards

AI/BI dashboards now support cross-filtering, which allows you to click on an element in one chart to filter and update related data in other charts.Cross-filtering allows users to interactively explore relationships and patterns across multiple visu...

cross-filter.gif
Community Articles
AI BI
Cross-filtering
Dashboards
reports
  • 4035 Views
  • 3 replies
  • 5 kudos
Latest Reply
SFDataEng
New Contributor III
  • 5 kudos

Yes, something like that could do. Reflections on the video/concept/comment are below.Summary:Option 1: Upon adding a cross-filter, it could work to add a short, top horizontal bar of cross-filters (which expands vertically if there are more than fit...

  • 5 kudos
2 More Replies
Nidhi_Patni
by New Contributor III
  • 1151 Views
  • 2 replies
  • 5 kudos

How We Built Robust Data Governance at Scale

In today's data-driven world, trust is currency—and that trust starts with quality data governed by strong principles. For one of our client, where we're on a mission to build intelligent enterprises with AI, data isn't just an asset—it's a responsib...

Nidhi_Patni_0-1753460966300.png Nidhi_Patni_5-1753461715636.png Nidhi_Patni_2-1753460966324.png Nidhi_Patni_3-1753460966330.png
  • 1151 Views
  • 2 replies
  • 5 kudos
Latest Reply
Dr-Sylvester
New Contributor III
  • 5 kudos

Great Article  @Nidhi_Patni 

  • 5 kudos
1 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels