- 2237 Views
- 2 replies
- 6 kudos
Data + AI Is Not the Future at Databricks. It’s the Present.
One thing becomes very clear when you spend time in the Databricks community: AI is no longer an experiment. It is already part of how real teams build, ship, and operate data systems at scale.For a long time, many organizations treated data engineer...
- 2237 Views
- 2 replies
- 6 kudos
- 10477 Views
- 3 replies
- 6 kudos
Getting Databricks Generative AI Engineer Associate (and What I Learned)
I just earned my Databricks Certified Generative AI Engineer Associate Certification, and in this post, I’m sharing the key tips, resources, and personal insights that helped me succeed. My certiticate from DatabricksNavigation:PreludeAbout the certR...
- 10477 Views
- 3 replies
- 6 kudos
- 6 kudos
Hey @devipriya , thanks for sharing your notes on how you found success with the certification(s). Appreciate you taking the time to pass along what worked for you. Cheers, Louis
- 6 kudos
- 1122 Views
- 2 replies
- 3 kudos
Change default workspace access to Consumer access
When Databricks One was launched, the default behaviour of the system-managed users group was a major issue. Since every new user is automatically added to users group, and that group traditionally came with "Workspace Access" entitlements, admins ha...
- 1122 Views
- 2 replies
- 3 kudos
- 338 Views
- 0 replies
- 3 kudos
Very short video on tip about Collect() method!!!
Here is a very short video on how to use collect() method with caution as it could overwhelm the cluster in some big data scenarios!!!! Now only in Spanish but I guess youtube will do the trick to translate into very correct English xDD
- 338 Views
- 0 replies
- 3 kudos
- 3774 Views
- 3 replies
- 6 kudos
A Governance-First Unified Namespace:Why Manufacturers Need Unity Catalog to Scale Industry 4.0 Data
The Industrial Data ChallengeManufacturing enterprises today operate with a fundamental paradox: they're drowning in data yet starving for insights. A typical plant generates terabytes of information daily across dozens of systems—from shop floor PLC...
- 3774 Views
- 3 replies
- 6 kudos
- 619 Views
- 0 replies
- 0 kudos
Great Expectations with Fabric and Databricks-Building Bulletproof Data Pipelines
In this article, I’ll walk you through transforming a basic PySpark notebook into a production-ready data pipeline with comprehensive quality checks using Great Expectations patterns in Microsoft Fabric and Databricks. We’ll start simple and progress...
- 619 Views
- 0 replies
- 0 kudos
- 666 Views
- 1 replies
- 1 kudos
Dashboards deployment
It is finally possible to deploy dashboards using DABS and change the catalog and schema. It is solving the biggest problem with bringing the dashboard to production. New parameters for the dashboard resource were added: dataset_catalog and dataset...
- 666 Views
- 1 replies
- 1 kudos
- 1 kudos
@Hubert-Dudek - This was really a much-awaited feature. Eventually DAB must cover all developed assets which requires promotion to higher environment to cover the CICD seamlessly.
- 1 kudos
- 1197 Views
- 1 replies
- 2 kudos
SQL Scripting - Procedural Logic in Databricks SQL (Without Leaving SQL)
We've all been there. You need to loop through a list of tables, apply some transformation, handle a few edge cases, and maybe catch an error or two. In most data platforms, that means switching to Python or Scala—even if 90% of your logic is just SQ...
- 1197 Views
- 1 replies
- 2 kudos
- 2 kudos
Correction: Preview is available on Databricks Runtime 16.3 and above.
- 2 kudos
- 4700 Views
- 0 replies
- 2 kudos
Databricks Serverless Compute: Performance, Cost, and Time-to-Value Explained
As data platforms mature, the focus is no longer just scalability—it is about speed, simplicity, and cost efficiency. Engineering teams want to deliver insights faster without managing infrastructure, while organizations want predictable costs and st...
- 4700 Views
- 0 replies
- 2 kudos
- 857 Views
- 1 replies
- 1 kudos
Window Functions in Metrics Views
The latest update for the first week of 2026 is the addition of window functions in Metrics View. In enterprises, there are always measures like cumulative sales or rolling forecast, so it is really important that we can use window functions in busin...
- 857 Views
- 1 replies
- 1 kudos
- 1 kudos
@Hubert-Dudek - Thanks for sharing these bite-size updates as these get lost in many release-notes. Keep it up.
- 1 kudos
- 349 Views
- 0 replies
- 2 kudos
Mix Shell with Python
Assigning the result of a shell command directly to a Python variable. It is my most significant finding in magic commands and my favourite one so far. Read about 12 magic commands in my blogs: - https://www.sunnydata.ai/blog/databricks-hidden-magic-...
- 349 Views
- 0 replies
- 2 kudos
- 667 Views
- 0 replies
- 3 kudos
Secure Credit Card Partner Enablement Using Databricks Clean Rooms
How Digital Payment Lending Platforms Can Collaborate with Banks Without Exposing Sensitive Data1. Business Context & Regulatory RealityIn 2020, large Indian fintech platforms faced a unique regulatory constraint: NBFC‑led digital platforms were not ...
- 667 Views
- 0 replies
- 3 kudos
- 430 Views
- 0 replies
- 1 kudos
Capture magic command
%%capture magic command not only suppresses cell output but also assigns it to a variable. You can later print cell output just by using the standard print() function #databricksRead about 12 magic commands in my blogs:- https://www.sunnydata.ai/blog...
- 430 Views
- 0 replies
- 1 kudos
- 488 Views
- 1 replies
- 3 kudos
Identifiers Everywhere
Last from "everywhere" improvements in Spark 4.1 / Runtime 18 is IDENTIFIER(). Lack of support for IDENTIFIER() in many places is a major pain, especially when creating things like Materialized Views or Dashboard Queries. Of course, we need to wait a...
- 488 Views
- 1 replies
- 3 kudos
- 3 kudos
Thanks for sharing @Hubert-Dudek !
- 3 kudos
- 908 Views
- 0 replies
- 3 kudos
Secret magic commands
Secret magic commands, there are a lot of them. Check my blogs to see which one can simplify your daily work. First one is %%writefile, which can be used to write a new file, for example, another notebook #databricks more magic commands:- https://dat...
- 908 Views
- 0 replies
- 3 kudos
| User | Count |
|---|---|
| 85 | |
| 75 | |
| 67 | |
| 61 | |
| 44 |
