Get Started Resources
Explore essential resources to kickstart your journey with Databricks. Access tutorials, guides, and...
0 Posts
Activity in Resources
- 2815 Views
- 11 replies
- 5 kudos
Improved Duplicate Match Handling in Delta MERGE Operations
In DBR 16.1+, we’ve improved functionality of MERGE operations where multiple rows of the source dataset match the same row of the target Delta table, but only one row matches the WHEN MATCHED condition. In the past, these operations would fail with...
- 2815 Views
- 11 replies
- 5 kudos
Latest Reply
- 5 kudos
@BamBam I tried it, but didn't get an error. What am I missing? See below: Sample example just adding some new values for insert. MERGE INTO test_data t USING (SELECT * FROM VALUES ("4", "A", 1), ("4", "D", 2), ...
- 5 kudos
- 110902 Views
- 22 replies
- 37 kudos
Metadata-Driven ETL Framework in Databricks (Part-1)
In modern data-driven enterprises, data flows like lifeblood through complex systems and repositories to drive decision-making and innovation. Each dataset, whether structured or unstructured, holds the potential to unlock insights and drive innovati...
- 110902 Views
- 22 replies
- 37 kudos
Latest Reply
- 37 kudos
This is a great article and I could see many benefits of using the Framework driven ETL. @Rjt_de any expected timeline on Part 2?
- 37 kudos
- 461 Views
- 0 replies
- 1 kudos
Evaluate Multi‑Turn Chatbots on Databricks with MLflow: A Step‑by‑Step Guide
Hi, I’m Debu. I spend a lot of my day building and stress‑testing LLM‑powered systems, and one lesson keeps coming back: if you don’t measure your agent’s behavior over an entire conversation, you’re flying blind. Below is the exact notebook pattern ...
- 461 Views
- 0 replies
- 1 kudos
- 289 Views
- 0 replies
- 0 kudos
Securing Sensitive Data - Using Encryption, Secrets, and User Context Filter on PII Data
Problem Statement Let us start with setting some context. The problem statement that we are solving here is kept a bit generic as the solution can be applied to any similar situations. Consider a Payroll datasource with PII data is ingested into Data...
- 289 Views
- 0 replies
- 0 kudos
- 397 Views
- 0 replies
- 0 kudos
Speeding up LLM inference by using model quantization in Databricks
The ever-increasing complexity of LLM models often comes at a steep cost: greater computational requirements, increased energy consumption, and slower inference times. Enter model quantization - a powerful technique that can substantially reduce mode...
- 397 Views
- 0 replies
- 0 kudos
- 16096 Views
- 1 replies
- 10 kudos
6 Migration Mistakes You Don’t Want To Make: Part 1
Congratulations! Your team has just signed a contract with Databricks and you’re ready to start your first project. Maybe. Statistically, most of you will need to migrate off your previous platform before you can start building something new. Or ma...
- 16096 Views
- 1 replies
- 10 kudos
Latest Reply
- 10 kudos
I agree, but I found more and wrote the first part, maybe it will be useful to someone
- 10 kudos
- 158 Views
- 0 replies
- 0 kudos
How to train a Convolutional Neural Network on Databricks with Tensorflow and Keras
Here is how to trained a lightweight Convolutional Neuronal Network (CNN) to detect pneumonia from chest X-rays pictures on Azure Databricks. I promise no LLMs, no hype, just real-world deep learning:1. Built it with TensorFlow & Keras on Databricks2...
- 158 Views
- 0 replies
- 0 kudos
- 1234 Views
- 0 replies
- 2 kudos
Business Intelligence in the Era of AI
AI is transforming BI by changing the way organizations manage, analyze and get insights from their data. Everyone is rushing to democratize access to data through LLMs. But without the right foundation, you can’t get the most from AI. Our new appro...
- 1234 Views
- 0 replies
- 2 kudos
- 3204 Views
- 0 replies
- 2 kudos
The Ultimate Guide to Chunking Strategies for RAG Applications with Databricks
Over the years, I have collaborated closely with ML engineering leaders across various industries, guiding them on how to make the right chunking strategy decisions for their Retrieval-Augmented Generation (RAG) use cases. One of the biggest challeng...
- 3204 Views
- 0 replies
- 2 kudos
- 709 Views
- 0 replies
- 2 kudos
Migrating ML Workload to Databricks
As machine learning (ML) workloads continue to grow in complexity and scale, organizations are looking for efficient and scalable solutions to manage their ML lifecycle. Databricks offers a powerful platform for ML workloads, providing scalability, s...
- 709 Views
- 0 replies
- 2 kudos
- 959 Views
- 0 replies
- 2 kudos
Building Trust in Data II: A Guide to Effective Data Testing Tactics
Introduction Building a reliable data pipeline goes beyond setting up a functional workflow — it requires meticulous testing to ensure data accuracy, integrity, and quality across every stage of the process. In this second part of our series on data ...
- 959 Views
- 0 replies
- 2 kudos
- 634 Views
- 0 replies
- 2 kudos
Building Trust in Data I: A Guide to Effective Data Testing Frameworks
Introduction When should I test my data? How should it be done? Which tools and testing methods are most effective? Who holds responsibility for data quality?” These are common questions that arise in discussions around data testing, often sparking m...
- 634 Views
- 0 replies
- 2 kudos
- 3427 Views
- 2 replies
- 4 kudos
Mastering RAG Chatbot Security: ACL and Metadata Filtering with Mosaic AI Vector Search
Imagine you’re running a company with multiple departments, like Finance, Legal, and HR. Each department has its own sensitive data—financial reports, legal contracts, and employee records—that need to stay private. Now, picture a star employee, a RA...
- 3427 Views
- 2 replies
- 4 kudos
Latest Reply
- 4 kudos
@s-udhaya and @jiayi-wu really excellent work, thank you for the write up. @Ben_H good question and thank you for providing your code. I used your above in place of the existing chain in the Chat History Extractor Chain command of the llm-rag-chatbo...
- 4 kudos
- 232 Views
- 0 replies
- 0 kudos
When Did the Data Go Wrong? Using Delta Lake Time Travel for Investigation in Databricks
I. IntroductionData pipelines are the lifeblood of modern data-driven organizations. However, even the most robust pipelines can experience unexpected issues: data corruption, erroneous updates, or sudden data drops. When these problems occur, quickl...
- 232 Views
- 0 replies
- 0 kudos
- 1740 Views
- 1 replies
- 2 kudos
Share Your Thoughts on Databricks & Get Rewarded!
We want to hear from you! Be among the first 20 people to share your experience with Databricks on G2 and receive a $50 gift card.Why Participate? Your Feedback Matters: It helps us innovate and improve.Quick and Easy: The review takes less than...
- 1740 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
Thank you all for your responses. We are closing this now!
- 2 kudos
