- 11263 Views
- 4 replies
- 2 kudos
Optimizing Costs in Databricks by Dynamically Choosing Cluster Sizes
Databricks is a popular unified data analytics platform known for its powerful data processing capabilities and seamless integration with Apache Spark. However, managing and optimizing costs in Databricks can be challenging, especially when it comes ...
- 11263 Views
- 4 replies
- 2 kudos
- 2 kudos
@Second Reply You’re right just printing out selected_pool isn’t enough to actually leverage dynamic cluster sizing at runtime. In practice, the value of selected_pool would feed directly into your Databricks cluster creation API or workflow automati...
- 2 kudos
- 1617 Views
- 2 replies
- 6 kudos
Meta Ads is now a native data source in Databricks Lakeflow Connect
Meta Ads is now a native data source in DatabricksDatabricks just announced a Meta Ads connector (Beta) powered by Lakeflow Connect, making it easy to ingest advertising data directly into Databricks—no custom APIs, no CSV exports, no brittle scripts...
- 1617 Views
- 2 replies
- 6 kudos
- 6 kudos
Have to test it. Confluence Lakeflow Connect is amazing, so I suspect that one also!
- 6 kudos
- 601 Views
- 1 replies
- 2 kudos
Flexible Node Types
Recently, it has not only become difficult to get a quota in some regions, but even if you have one, it doesn't mean that there are available VMs. Even if you have a quota, you may need to move your bundles to a different subscription when different ...
- 601 Views
- 1 replies
- 2 kudos
- 2 kudos
Thanks for sharing, @Hubert-Dudek. This is a common challenge users face, and flexible node types can significantly improve compute launch reliability.
- 2 kudos
- 490 Views
- 0 replies
- 1 kudos
DABs: Referencing Your Resources
From hardcoded IDs, through lookups, to finally referencing resources. I think almost everyone, including me, wants to go through such a journey with Databricks Asset Bundles. #databricks In the article below, I am looking at how to reference a resou...
- 490 Views
- 0 replies
- 1 kudos
- 541 Views
- 0 replies
- 1 kudos
Confluence Lakeflow Connector
Incrementally upload data from Confluence. I remember there were a few times in my life when I spent weeks on it. Now, it is incredible how simple it is to implement it with Lakeflow Connect. Additionally, I love DABS's first approach for connectors,...
- 541 Views
- 0 replies
- 1 kudos
- 2021 Views
- 4 replies
- 4 kudos
From Learning to Enablement: My 2025 Databricks Journey
Celebrating platform capabilities, community impact, and responsible adoptionIn 2025, my Databricks journey evolved from mastering features to empowering outcomes.What became clear this year is that Databricks isn’t just a powerful platform — it’s a ...
- 2021 Views
- 4 replies
- 4 kudos
- 4 kudos
Nice write-up @Lakshmipriya_Na , I really like this framing. The “Builder” vs “Strategist” distinction maps almost perfectly to how Databricks shows up in the real world. You can move fast and iterate in notebooks, but the same Lakehouse naturally n...
- 4 kudos
- 479 Views
- 0 replies
- 1 kudos
Databricks Advent Calendar 2025 #23
Our calendar is coming to an end. One of the most significant innovations of last year is Agent Bricks. We received a few ready-made solutions for deploying agents. As the Agents ecosystem becomes more complex, one of my favourites is the Multi-Agent...
- 479 Views
- 0 replies
- 1 kudos
- 449 Views
- 0 replies
- 1 kudos
Databricks Advent Calendar 2025 #22
During the last two weeks, five new Lakeflow Connect connectors were announced. It allows incremental ingestion of the data in an easy way. In the coming weeks, there will be more announcements about Lakeflow Connect, and we can expect Databricks to ...
- 449 Views
- 0 replies
- 1 kudos
- 495 Views
- 0 replies
- 1 kudos
Databricks Advent Calendar 2025 #21
Your stream can have a state, and now, with TransformWithStateInPandas, it’s easy to manage - you can handle things like initial state, deduplication, recovery, etc., with the 2025 improvements.
- 495 Views
- 0 replies
- 1 kudos
- 642 Views
- 0 replies
- 1 kudos
Databricks Advent Calendar 2025 #20
As Unity Catalog becomes an enterprise catalog, bring-your-own lineage is one of my favorite features.
- 642 Views
- 0 replies
- 1 kudos
- 565 Views
- 0 replies
- 1 kudos
Databricks Advent Calendar 2025 #19
In 2025, Metrics Views are becoming the standard way to define business logic once and reuse it everywhere. Instead of repeating complex SQL, teams can work with clean, consistent metrics.
- 565 Views
- 0 replies
- 1 kudos
- 657 Views
- 1 replies
- 2 kudos
Databricks Advent Calendar 2025 #18
Automatic file retention in the autoloader is one of my favourite new features of 2025. Automatically move cloud files to cold storage or just delete.
- 657 Views
- 1 replies
- 2 kudos
- 2 kudos
Thanks for sharing @Hubert-Dudek ! That's a really great feature. It simplified a lot data maintenance process at one of my clients
- 2 kudos
- 2088 Views
- 3 replies
- 22 kudos
Training That Scales: How Databricks Adoption Really Takes Hold
I teach Databricks to all sorts of folks, coders, managers—everyone. What’s wild is how two companies with the same setup can have totally different experiences. Usually, it’s not the tech itself that’s the issue, but how people see it. Databricks tr...
- 2088 Views
- 3 replies
- 22 kudos
- 22 kudos
As someone who benefited from Louis Training, I can attest that it makes a difference to constantly keep up to date and work on the foundation of understanding.Especially when things move so fast as in our industry, the time to reflect and improve pa...
- 22 kudos
- 9673 Views
- 8 replies
- 7 kudos
Cross-filtering for AI/BI dashboards
AI/BI dashboards now support cross-filtering, which allows you to click on an element in one chart to filter and update related data in other charts.Cross-filtering allows users to interactively explore relationships and patterns across multiple visu...
- 9673 Views
- 8 replies
- 7 kudos
- 7 kudos
There does appear to now be a list of capsules indicating the filters applied along the top of Databricks AI/BI Dashboards. The capsules appear to include filter-selectors and also cross-filters added by clicking charts.Also, there is a now "Reset t...
- 7 kudos
- 427 Views
- 0 replies
- 2 kudos
Databricks Advent Calendar 2025 #17
Replacing records for the entire date with newly arriving data for the given date is a typical design pattern. Now, thanks to simple REPLACE USING in Databricks, it is easier than ever!
- 427 Views
- 0 replies
- 2 kudos
| User | Count |
|---|---|
| 85 | |
| 75 | |
| 72 | |
| 60 | |
| 43 |
