- 18446 Views
- 3 replies
- 7 kudos
Resolved! Data Quality with PySpark and Great Expectations on Databricks
Data governance is one of the most important pillars in any modern architecture. When building pipelines that process data at scale, ensuring data quality is not just a best practice—it is a critical necessity.Tools like Great Expectations (GX) were ...
- 18446 Views
- 3 replies
- 7 kudos
- 4771 Views
- 0 replies
- 1 kudos
Tracking Query History and Optimizing Queries in Databricks
Optimizing queries in Databricks isn’t just about adding indexes or tweaking SQL syntax — it’s about visibility. You can’t improve what you can’t measure. Fortunately, Databricks provides rich telemetry around query history that you can use to analyz...
- 4771 Views
- 0 replies
- 1 kudos
- 4823 Views
- 1 replies
- 4 kudos
[Blog] Databricks Serverless vs Classic: Who Wins the Cost Sprint?
Hi everyone! I wanted to share with you a post I wrote on Medium a while ago — it’s still very useful if you want to understand how to properly calculate Databricks cluster costs and get a realistic view of the differences: Databricks Serverless vs C...
- 4823 Views
- 1 replies
- 4 kudos
- 4 kudos
Really interesting topic I'll take a look when possible Always interested in improving performance and saving cloud costs. Thanks for sharing.
- 4 kudos
- 2763 Views
- 2 replies
- 3 kudos
Resolved! 🚀 Boost Databricks Performance ✅ Lazy Evaluation and Spark DataFrame Caching ☕
Published new video in my recently created youtube channel about one of my favorite topics: performance. So, here is a new video whose goal is to explain clearly what lazy evaluation is and how to use caching to boost performance. https://studio.y...
- 2763 Views
- 2 replies
- 3 kudos
- 3 kudos
Not sure if previous link works fine, here is the correct one https://youtu.be/pLGAr1VXQSQ?si=QaKzaNfzrNl_0Tv9
- 3 kudos
- 3291 Views
- 4 replies
- 3 kudos
Resolved! Introduction to Databricks for Beginners Video
Here is the first episode in ENGLISH VERSION of a list of simple videos on Introduction to Databricks for beginners:Introduction to Databricks for Beginners - Episode 1 It contains previous and basic concepts to master before moving forward with Data...
- 3291 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @Coffee77 ,Thanks for sharing with us. It looks promising. Can't wait for another episode
- 3 kudos
- 4686 Views
- 4 replies
- 7 kudos
Resolved! Databricks for RAG: Build, Run, Evaluate
What is RAG?RAG (Retrieval-Augmented Generation) on Databricks refers to building and running AI applications that combine:Retrieval systems (like vector databases or search over documents)Generative AI models (such as LLMs like GPT)within the Databr...
- 4686 Views
- 4 replies
- 7 kudos
- 2791 Views
- 4 replies
- 3 kudos
Resolved! Introduction to Databricks 🇪🇸
Here is the first episode of a serie of simple videos on Introduction to Databricks for beginners :https://youtu.be/kvglz79Ob-M?si=KnyCH74_HQ8jiO7SIt contains previous and basic concepts to master before moving forward with Databricks.
- 2791 Views
- 4 replies
- 3 kudos
- 3 kudos
English version is ready : INTRODUCTION to DATABRICKS in English - Episode 1 ‌
- 3 kudos
- 3399 Views
- 1 replies
- 3 kudos
Databricks Free Edition: The Announcement from Data + AI Summit 2025
The Data + AI Summit 2025 delivered several groundbreaking announcements, but none were more democratizing than the launch of the new Databricks Free Edition. Announced alongside a massive $100 million investment in training, this new offering provid...
- 3399 Views
- 1 replies
- 3 kudos
- 3 kudos
Nicely outlined, @nayan_wylde!
- 3 kudos
- 51394 Views
- 3 replies
- 2 kudos
Databricks Community Edition Login - Sign Up/Sign In/Forgot Password
Sign Up Go to https://www.databricks.com/try-databricksFill in the 2 steps box on the right hand side Note - It is important to select the Personal use section in the above step. Sign In Enter your details here https://accounts.cloud.databricks.com/...
- 51394 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi,I cannot signup to Community edition. When I try to sign up using this link https://www.databricks.com/try-databricks it first shows this pop up, non of these two options allows me to signup for community edition. I don't find option 'get started...
- 2 kudos
- 2122 Views
- 5 replies
- 7 kudos
Resolved! Generating a PostgreSQL Table Schema for ETL in Databricks
In a data migration project, I needed to generate the schema of a PostgreSQL table to use in my ETL process. I’d like to share the code snippet in case someone else needs it one day:from pyspark.sql import SparkSession import json import os from typi...
- 2122 Views
- 5 replies
- 7 kudos
- 3316 Views
- 1 replies
- 0 kudos
Resolved! Automating Notebook Documentation in Databricks with LLMs
In one of my projects, I needed to generate structured documentation for an entire directory of Databricks notebooks.This solution uses the Databricks Workspace API together with a Serving Endpoint (LLM) to automatically create HTML documentation for...
- 3316 Views
- 1 replies
- 0 kudos
- 0 kudos
Suggestions are always welcome — I hope this helps anyone looking to automate notebook documentation in Databricks.
- 0 kudos
- 924 Views
- 2 replies
- 7 kudos
Data Security at the level of columns or rows or Data masking
Hi everyone, I'm currently going through the Data Analyst learning path. I've just learned about Dynamic Views and I wanted to share the article on them: https://docs.databricks.com/aws/en/views/dynamic#before-you-begin There are some limitations on ...
- 924 Views
- 2 replies
- 7 kudos
- 7 kudos
@BS_THE_ANALYST Cool stuff, right! Have read about Attribute-based Access Control (ABAC) yet? Check it out: https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/ Let me know what you think. Cheers, Louis.
- 7 kudos
- 1616 Views
- 0 replies
- 2 kudos
Databricks Asset Bundles with Python!
Databricks Asset Bundles now support Python!If you’re a Python fan, you can define jobs and pipelines in Python (or keep YAML) with DABs. You can create jobs from simple metadata, modify them during deployment with mutators, and convert existing jobs...
- 1616 Views
- 0 replies
- 2 kudos
- 4427 Views
- 0 replies
- 1 kudos
Lakehouse vs. Lakehouse Federation - Bridging the Next Evolution in Data Platforms
Over the past few years, the Lakehouse architecture has become the gold standard for managing modern data workloads. By combining the low-cost storage of data lakes with the reliability and performance of data warehouses, Lakehouses have redefined ho...
- 4427 Views
- 0 replies
- 1 kudos
- 2674 Views
- 0 replies
- 2 kudos
Understanding Modern Databricks Warehousing for the AI era: A Beginner’s Guide
IntroductionIn the current Gen AI buzz, most conversations focus on RAG for unstructured documents. But there’s another equally important challenge — making sense of structured data at scale.This is where tools like Databricks Genie step in, enablin...
- 2674 Views
- 0 replies
- 2 kudos
| User | Count |
|---|---|
| 85 | |
| 75 | |
| 72 | |
| 60 | |
| 43 |
