- 5581 Views
- 4 replies
- 7 kudos
Resolved! Databricks for RAG: Build, Run, Evaluate
What is RAG?RAG (Retrieval-Augmented Generation) on Databricks refers to building and running AI applications that combine:Retrieval systems (like vector databases or search over documents)Generative AI models (such as LLMs like GPT)within the Databr...
- 5581 Views
- 4 replies
- 7 kudos
- 16236 Views
- 5 replies
- 8 kudos
Resolved! Databricks Machine Learning Professional Preparation
Recently I earned the Databricks Machine Learning Professional certification and wanted to share my study journey. Before the exam, I worked on a project as a data engineer alongside data scientists (ML models, LLMs, MLflow). That led me to build a p...
- 16236 Views
- 5 replies
- 8 kudos
- 8 kudos
Thanks a lot, my friend @BS_THE_ANALYST ! Really glad you found it useful . I’m sure when you dive into ML later this year, you’ll do awesome things with it. Appreciate the kind words about the project — means a lot! All the best to you too, and let’...
- 8 kudos
- 3291 Views
- 4 replies
- 3 kudos
Resolved! Introduction to Databricks 🇪🇸
Here is the first episode of a serie of simple videos on Introduction to Databricks for beginners :https://youtu.be/kvglz79Ob-M?si=KnyCH74_HQ8jiO7SIt contains previous and basic concepts to master before moving forward with Databricks.
- 3291 Views
- 4 replies
- 3 kudos
- 3 kudos
English version is ready : INTRODUCTION to DATABRICKS in English - Episode 1
- 3 kudos
- 3895 Views
- 1 replies
- 3 kudos
Databricks Free Edition: The Announcement from Data + AI Summit 2025
The Data + AI Summit 2025 delivered several groundbreaking announcements, but none were more democratizing than the launch of the new Databricks Free Edition. Announced alongside a massive $100 million investment in training, this new offering provid...
- 3895 Views
- 1 replies
- 3 kudos
- 3 kudos
Nicely outlined, @nayan_wylde!
- 3 kudos
- 57834 Views
- 3 replies
- 2 kudos
Databricks Community Edition Login - Sign Up/Sign In/Forgot Password
Sign Up Go to https://www.databricks.com/try-databricksFill in the 2 steps box on the right hand side Note - It is important to select the Personal use section in the above step. Sign In Enter your details here https://accounts.cloud.databricks.com/...
- 57834 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi,I cannot signup to Community edition. When I try to sign up using this link https://www.databricks.com/try-databricks it first shows this pop up, non of these two options allows me to signup for community edition. I don't find option 'get started...
- 2 kudos
- 2517 Views
- 5 replies
- 7 kudos
Resolved! Generating a PostgreSQL Table Schema for ETL in Databricks
In a data migration project, I needed to generate the schema of a PostgreSQL table to use in my ETL process. I’d like to share the code snippet in case someone else needs it one day:from pyspark.sql import SparkSession import json import os from typi...
- 2517 Views
- 5 replies
- 7 kudos
- 3660 Views
- 1 replies
- 0 kudos
Resolved! Automating Notebook Documentation in Databricks with LLMs
In one of my projects, I needed to generate structured documentation for an entire directory of Databricks notebooks.This solution uses the Databricks Workspace API together with a Serving Endpoint (LLM) to automatically create HTML documentation for...
- 3660 Views
- 1 replies
- 0 kudos
- 0 kudos
Suggestions are always welcome — I hope this helps anyone looking to automate notebook documentation in Databricks.
- 0 kudos
- 1062 Views
- 2 replies
- 7 kudos
Data Security at the level of columns or rows or Data masking
Hi everyone, I'm currently going through the Data Analyst learning path. I've just learned about Dynamic Views and I wanted to share the article on them: https://docs.databricks.com/aws/en/views/dynamic#before-you-begin There are some limitations on ...
- 1062 Views
- 2 replies
- 7 kudos
- 7 kudos
@BS_THE_ANALYST Cool stuff, right! Have read about Attribute-based Access Control (ABAC) yet? Check it out: https://docs.databricks.com/aws/en/data-governance/unity-catalog/abac/ Let me know what you think. Cheers, Louis.
- 7 kudos
- 1767 Views
- 0 replies
- 2 kudos
Databricks Asset Bundles with Python!
Databricks Asset Bundles now support Python!If you’re a Python fan, you can define jobs and pipelines in Python (or keep YAML) with DABs. You can create jobs from simple metadata, modify them during deployment with mutators, and convert existing jobs...
- 1767 Views
- 0 replies
- 2 kudos
- 4967 Views
- 0 replies
- 1 kudos
Lakehouse vs. Lakehouse Federation - Bridging the Next Evolution in Data Platforms
Over the past few years, the Lakehouse architecture has become the gold standard for managing modern data workloads. By combining the low-cost storage of data lakes with the reliability and performance of data warehouses, Lakehouses have redefined ho...
- 4967 Views
- 0 replies
- 1 kudos
- 3022 Views
- 0 replies
- 2 kudos
Understanding Modern Databricks Warehousing for the AI era: A Beginner’s Guide
IntroductionIn the current Gen AI buzz, most conversations focus on RAG for unstructured documents. But there’s another equally important challenge — making sense of structured data at scale.This is where tools like Databricks Genie step in, enablin...
- 3022 Views
- 0 replies
- 2 kudos
- 1322 Views
- 0 replies
- 0 kudos
Apache Spark 4.0 — Big Data Engineering!
The latest Spark 4.0 release delivers powerful enhancements across SQL, Python, streaming, and connectivity — all aimed at making big data workloads more efficient, reliable, and developer-friendly.With Databricks Runtime 17.0, these capabilities are...
- 1322 Views
- 0 replies
- 0 kudos
- 3477 Views
- 0 replies
- 1 kudos
Pipelines to Prompts: Getting started with Databricks and AWS
NAVIGATION:Why Data EngineeringThe Role of Data Engineering in GenAIWhat is Databricks? Unifying Data and AI on One PlatformDatabricks on AWS: A Full-Stack Platform for GenAIHands-On ExerciseFuture-Proofing: Why Data + AI Skills Matter Now More Than ...
- 3477 Views
- 0 replies
- 1 kudos
- 9203 Views
- 1 replies
- 3 kudos
Understanding Liquid Clustering in Databricks - The Next Evolution in Data Optimisation
In the world of big data, organising data smartly is just as important as collecting it. When working with large datasets in Databricks using Delta Lake, how your data is stored and ordered can greatly impact performance, especially for queries. Trad...
- 9203 Views
- 1 replies
- 3 kudos
- 3 kudos
Great post, Rahul! You’ve nailed the key trade-offs perfectly. The Appeal: LC is “set it and forget it” data management—no more manual OPTIMIZE jobs or performance firefighting. The Reality Check: Single-column clustering works great for high-cardina...
- 3 kudos
- 2498 Views
- 2 replies
- 0 kudos
Recommendations for Designing Cluster Policies Across Dev/QA/Prod Environments for DE and DA teams
Hi Community,We are working on implementing Databricks cluster policies across our organization and are seeking advice on best practices to enforce governance, security, and cost control across different environments.We have two main teams using Data...
- 2498 Views
- 2 replies
- 0 kudos
- 0 kudos
I just want to confirm one more thing here is that me as admin managing the cluster creation and no user will have access to create them me know how the cluster policies help me in this perspective.
- 0 kudos
| User | Count |
|---|---|
| 85 | |
| 77 | |
| 73 | |
| 60 | |
| 43 |
