Explore in-depth articles, tutorials, and insights on data analytics and machine learning in the Databricks Technical Blog. Stay updated on industry trends, best practices, and advanced techniques.
SQL is a first-class language in SDP
SQL is a first-class language in Spark Declarative Pipelines (SDP). You can build, test, and ship an entire pipeline in SQL. This makes it a powerful tool for thos...
Building a Production LangGraph Agent on Databricks: The NorthStar Brand Copilot
How we built and deployed an end-to-end CPG AI agent on Databricks using LangGraph, MCP servers, Lakebase memory, Genie...
Every time you copy lakehouse data into an app-side database, you create another system to maintain, another copy to secure, and another pipeline to debug at 2 AM. But that is exactly what teams do wh...
Today, we are introducing Beta support for Kafka-compatible producer API support in Zerobus Ingest. This is an incredible milestone for Zerobus, given the wide ecosystem built around Kafka.
Take a ...
Introduction
This is the final part of a three-part series on concurrent load testing of Databricks SQL warehouses with Apache JMeter. In Part 1 we externalized our SQL queries and parameters, and in ...
Introduction
This is the second part of a three-part series on concurrent load testing of Databricks SQL warehouses with Apache JMeter. In Part 1, we externalized our SQL queries and parameter values ...
Introduction
A single query that runs quickly in the SQL editor may not fully reveal how a warehouse behaves when many users query it concurrently. Under concurrent load, queries queue, contend for re...
Table of Contents
Understanding Metric Views
What are Metric Views?UI + YAML: Two Layers, One DefinitionWhen to Use the UI and YAML
Building Metric Views
Visual Data ModelingMeasure BuilderPreview Beh...
Databricks AI/BI dashboards ship with a solid set of visualization types. But every so often you want something the point-and-click widgets can't quite do.
That's what custom visualizations (currentl...
It is Friday afternoon. An upstream team ships a harmless-looking change: they add a region field to the events your service already emits, and somewhere along the way, a sensor starts reporting temp...