TL;DR Until recently, a Spark Declarative Pipeline’s (SDP) jurisdiction ended where the lakehouse ended. You could land, clean, and curate data into governed tables in Unity Catalog, and that was the ...
Does your “open source” architecture actually deliver freedom? For an enterprise, lock-in isn’t a binary choice; it is a spectrum of exit costs that must be weighed against total TCO—specifically the ...
The Lakeflow Spark Declarative Pipelines (SDP) Kafka sink is now generally available, turning your declarative pipeline into a unified engine for both data ingestion and real-time egress. You can now ...
Taking the proven MLOps deploy-code pattern and adapting it for GenAI agents: evaluation-driven CI/CD, a native observability loop, and open OpenTelemetry integration. From MLOps to AgentOps Most data...
Most teams I work with don't struggle with the AI part of MLOps. They struggle with the boring parts: getting code from a notebook into production without breaking it, knowing when a model has gone st...
Build a sub-second operational dashboard on live streaming data, starting with a live Kafka stream This blog is a full end-to-end walkthrough for practitioners on how to build real-time applications u...
As organizations scale their AI initiatives on Databricks, a common pattern emerges: one team builds and maintains a vector store, while other teams across different workspaces need to query it. Perha...
Unified stack for geospatial workloads Every day, hundreds of thousands of vessels broadcast their position, speed and heading via AIS. That data stream is a goldmine: commodity traders can sharpen ET...
Getting a model behind a REST endpoint is the easy part; however keeping it fast, affordable, and stable once production traffic shows up is where teams spend their real time. Databricks Model Serving...
Introduction For enterprise groups deploying Azure Databricks, the Azure Tenant — Databricks Account relationship is a foundational architectural decision. It shapes data isolation between subsidiarie...
