Dive into a collaborative space where members like YOU can exchange knowledge, tips, and best practices. Join the conversation today and unlock a wealth of collective wisdom to enhance your experience and drive success.
atabricks SQL 2026: From SQL Analytics to an AI-Ready Data PlatformThe Databricks SQL 2026 release cycle is not just about adding more SQL functions.It reflects a much bigger shift:Databricks SQL is evolving from a query and reporting layer into a fo...
IntroductionIoT devices generate a large amount of data every day. In a perfect scenario, all this data arrives on time and our ingestion pipeline processes it using incremental loads. But in real-world IoT systems, this does not always happen.A devi...
Building a customer 360 requires connecting customer records across disparate data sets and establishing common customer identities. The Customer Entity Resolution Solution Accelerator shows how to build that foundation by translating customer attrib...
Answer firstA medallion architecture over SAP data is the same three layers everyone draws, with three jobs that are specific to the source. Bronze has to preserve deletes. Silver has to rebuild a business document that lives in five tables. Gold has...
SummaryThis post walks through the architecture of The Third Eye, a continuous AI health and governed ModelOps control plane built entirely on native Databricks capabilities. The core constraint: it never recomputes a metric Databricks already comput...
I ran the same 24 coding tasks through three arms: fixed Haiku, fixed Opus and Databricks-managed selection. The managed-selection arm passed all 24 tasks. Its 24 routing records also said task profiling was unavailable.That is not a contradiction. T...
In my 13 years of architecting data platforms—from Retail to Industrial Gas Power—I’ve learned one universal truth: A fast pipeline that delivers bad data is just a liability.As we move toward Declarative Pipelines (DLT), the role of the Architect sh...
If you use ChatGPT Work, Claude Cowork or Cursor Projects, much of Omnigent will look familiar.All of these products now organize work around projects, files and persistent context. They can run tasks for longer periods, connect to external tools and...
As data platforms move toward more declarative and incremental processing, one question comes up frequently when building pipelines in Databricks:Should I use a Materialized View or a Streaming Table?Both are important building blocks in Lakeflow pip...
Building an Explainable Semantic Intelligence Platform on the Databricks LakehouseWe are building Nexa, a semantic intelligence platform for Databricks that turns a raw Lakehouse into a continuously maintained, explainable system that can compile nat...
Subscriber attrition can be difficult to anticipate. The Subscriber Churn Prediction Solution Accelerator shows how to analyze behavioral data, identify subscribers at increased risk of cancellation, and use machine learning to estimate churn likelih...
Databricks Training & Certifications
Learn Databricks Lakeflow
Unified data engineering: ingest, orchestrate, and build pipelines on one platform.
Ingest Orchestrate No-code ETL
Data engineering on Databricks com...
My son asked me one evening whether any asteroids were going to hit Earth. Thedata to answer him exists — NASA publishes it daily, for free — but it lives inJSON behind API keys, in units like astronomical units and X-ray flux classes,built for peopl...
Hi @sudiptob-DA,This is a fantastic use case for the Genie Agent. You’ve perfectly illustrated the most important lesson in modern AI-augmented data platforms: the quality of the natural language experience is entirely dependent on the quality of the...
Disclosure: I work at Onibex, the company that builds OneConnect, which is used as the worked example in this post. This is about the engineering approach, not a product pitch.Design of Databricks Pipelines for real-time data transformation to genera...
Product onboarding can be complex and time-consuming when teams work with inconsistent data formats, inaccurate information and multiple stakeholders. The Simplifying Product Onboarding With Generative AI Solution Accelerator shows how generative AI ...