- 345 Views
- 2 replies
- 5 kudos
Databricks Advent Calendar 2025 #3
One of the biggest gifts is that we can finally move Genie to other environments by using the API. I hope DABS comes soon.
- 345 Views
- 2 replies
- 5 kudos
- 149 Views
- 0 replies
- 0 kudos
Fast-Track Your Databricks Development: Using Widgets for Test and Production Modes
Test Mode Pattern with Databricks Widgets - Demo What's IncludedThis package contains comprehensive documentation and examples for implementing the Test Mode Pattern in Databricks notebooks using widgets. This pattern enables fast development cycles ...
- 149 Views
- 0 replies
- 0 kudos
- 315 Views
- 0 replies
- 1 kudos
Your Coworkers Don't Get Databricks? Here's How to Fix That.
We've all been there. You're excited about the lakehouse, you see the value clear as day, and then you try explaining it to a coworker and their eyes glaze over. Slides don't cut it. Documentation links get ignored. What actually works? Showing them...
- 315 Views
- 0 replies
- 1 kudos
- 1642 Views
- 4 replies
- 6 kudos
The Medallion Architecture: Why Data Layers Matter for Modern Organisations
In today’s data-driven world, organisations are drowning in information. From customer transactions and IoT sensor readings to social media interactions and operational logs, the volume and variety of data continue to grow exponentially. Yet many org...
- 1642 Views
- 4 replies
- 6 kudos
- 6 kudos
Thank you, @Louis_Frolio ! My next post is about Data Governance with Unity Catalog, stay tuned!!
- 6 kudos
- 5315 Views
- 5 replies
- 0 kudos
Unity cataog
Disaster recovery is possible in Unity catalog now?Means, for data level, we have enabled with geo redundancy, what about the objects, permissions, an other components in Unity catalog ? Can we restore the unity catalog metadata in another region ?
- 5315 Views
- 5 replies
- 0 kudos
- 0 kudos
Official product release in development will be available as PrPr in a few months.
- 0 kudos
- 217 Views
- 0 replies
- 1 kudos
Managing Databricks CLI Versions in Your DAB Projects
If you are going with DABS into a production environment, a CLI version is considered best practice. Of course, you need to remember to bump it up from time to time. Learn more: - https://databrickster.medium.com/managing-databricks-cli-versions-i...
- 217 Views
- 0 replies
- 1 kudos
- 331 Views
- 0 replies
- 1 kudos
Distributed ML on Databricks Serverless
You can now run distributed ML (Spark MLlib in Python, Optuna tuning, MLflow Spark, Joblib Spark) on serverless notebooks/jobs and on standard clusters, not just dedicated ML clusters.It reuses the same Unity Catalog + Lakeguard stack you already use...
- 331 Views
- 0 replies
- 1 kudos
- 1412 Views
- 0 replies
- 1 kudos
Why Your Delta Lake MERGE Takes Forever (And How to Fix It)
Last month, our nightly CDC pipeline started timing out. What used to complete in 20 minutes was now crawling past the 4-hour mark—and failing. The culprit? A MERGE statement against a 2.3TB Delta table with 800 million rows that had grown steadily o...
- 1412 Views
- 0 replies
- 1 kudos
- 384 Views
- 3 replies
- 0 kudos
Real-Time SQL Server to Databricks Pipeline Using Debezium, Kafka, and Delta Lake
Check this medium article https://medium.com/@bijumathewt/real-time-sql-server-to-databricks-pipeline-using-debezium-kafka-and-delta-lake-26c3e191ce51?postPublishedType=initial
- 384 Views
- 3 replies
- 0 kudos
- 0 kudos
would you not be able to use Lakeflow Managed connector for SQL Server in this case?
- 0 kudos
- 193 Views
- 0 replies
- 2 kudos
🚀 Automating Databricks Unity Catalog Access with AI Agents
Tired of complex governance workflows? We’ve successfully combined the power of multi-agent AI with state-of-the-art orchestration tools to automate Databricks Unity Catalog (UC) access management!This isn’t just a basic script — it’s an intelligent,...
- 193 Views
- 0 replies
- 2 kudos
- 130 Views
- 0 replies
- 0 kudos
Real-World LangGraph + CrewAI Application:Intelligent Databricks Alert System with AI Prioritization
https://medium.com/@bijumathewt/real-world-langgraph-crewai-application-intelligent-databricks-alert-system-with-ai-7ba36e23f7a2You will love above article
- 130 Views
- 0 replies
- 0 kudos
- 152 Views
- 0 replies
- 0 kudos
Real-World LangGraph + CrewAI Application: Intelligent Databricks Alert System with AI Prioritizatio
You will love this article. Watch it. https://medium.com/@bijumathewt/real-world-langgraph-crewai-application-intelligent-databricks-alert-system-with-ai-7ba36e23f7a2
- 152 Views
- 0 replies
- 0 kudos
- 1370 Views
- 0 replies
- 1 kudos
Delta Lake 4.0 in the Real World
Delta Lake 4.0 is the next major open-source release aligned with Spark 4.x, adding first-class Variant for semi-structured data, safer Type Widening, improved DROP FEATURE, better transaction log handling, and a new multi-engine story via Delta Kern...
- 1370 Views
- 0 replies
- 1 kudos
- 373 Views
- 1 replies
- 3 kudos
PDFs to Production
Databricks just solved a huge problem - unlocking the value from unstructured data. One of the biggest challenges enterprises face when scaling agents is access to unstructured data. Nearly 80% of enterprise knowledge is trapped in PDFs, reports, and...
- 373 Views
- 1 replies
- 3 kudos
- 3 kudos
Thanks for sharing, @venkat-raghavan! ai_parse_document turns documents into governed, queryable assets, it's definitely the puzzle piece all needed.
- 3 kudos
- 365 Views
- 0 replies
- 2 kudos
Databricks Managed MCP Servers
This is super amazing and super powerful -https://docs.databricks.com/aws/en/generative-ai/mcp/managed-mcpEssentially we you can use Databricks managed MCP servers to seamlessly integrate Databricks features into any AI Agent. That's the multiplier ...
- 365 Views
- 0 replies
- 2 kudos
