None of the new features matters if your platform goes down when a region or cloud has an outage. Cross-region and cross-cloud recovery is non-negotiable. This is key for any platform hosting applications, as it enables the components mentioned above...
As the Lakehouse becomes your primary data and application platform and everything is consolidated there, you will occasionally need to push that data outwards. For example, to run a marketing campaign in Google Ads. #databricks #DataAISummit https:/...
Once you have that single platform holding all your company data, you need a knowledge graph, ideally created automatically, to build a context layer on top of it. Genie Ontology. #databricks #DataAISummit https://databrickster.medium.com/my-favorite...
Heads up! Databricks AI Search has replaced Vector Search. The rebranding reflects a shift in focus from embeddings to a more flexible and versatile search solution that supports hybrid keyword-similarity search and full-text keyword search. This cha...
Once your apps are hosted in Databricks, you need one more element... Customer data lake If you need to run campaigns and maintain master records, a customer data lake becomes essential as it allows you to build your own solution (the premise: SaaS i...
In this video, you'll learn how to use Databricks Visual Data Prep to clean, transform, and enrich data using a no-code, AI-assisted visual interface for building production-ready data pipelines. Whether you're a data engineer, data analyst, or just...
Fast reads and fast writes, thanks to Lakebase, including writes to open formats. It creates a unified platform both for analytics and operational use cases. #databricks #DataAISummit https://databrickster.medium.com/my-favorite-announcements-from-th...
As data engineering teams scale out lakehouses and cloud data warehouses, a silent platform killer inevitably creeps in: runaway query costs.In a distributed environment like Databricks, a single unoptimized query whether it’s an accidental Cartesian...
As someone who spends hours every day building notebooks, debugging pipelines, and experimenting with AI agents in Databricks, there was always one thought in the back of my mind.“Why can’t Genie Code have a full-screen workspace where I can manage e...
Managed Tables, External Tables, Foreign Tables and... .. Catalog Managed Tables. If you are databricks user, you need to know why they are the best of all! https://databrickster.medium.com/catalog-commits-make-your-managed-delta-layer-safer-and-more...
This is included with the Databricks managed table; there is no separate catalog managed table.
The rapid evolution of AI agents has transformed how organizations automate tasks, generate insights, and accelerate software development. However, as teams adopt multiple AI models, frameworks, and agent orchestration tools, managing these systems e...
As organizations continue to scale their data platforms, managing consistency, governance, and transactional integrity across multiple datasets has become increasingly challenging. While Delta Lake has long provided robust ACID transactions at the ta...
In this video, I'm going to show you how to automate resource creation in Azure Databricks using the Databricks SDK for Python. If you've ever found yourself manually creating catalogs, schemas, and volumes in the Databricks UI repeatedly, this video...
Databricks Apps run 24/7 with no native scale-to-zero. ~$350/mo per app (720 hrs), but internal dashboards/admin tools only get used a few hours/day. You pay for 100%, use ~15%.We had a test app run idle for 69 days before anyone noticed. ~$800 waste...
Seems there is so much confusion on the upcoming Genie pricing.Databricks Genie pricing, made simple:Free monthly usage stays Only usage beyond it is billed (DBUs)Paid starts July 6, 2026Admins set budgets to cap spendGenie did NOT become paid-only. ...
