- 18662 Views
- 6 replies
- 3 kudos
Editing value of widget parameter within notebook code
I have a notebook with a text widget where I want to be able to edit the value of the widget within the notebook and then reference it in SQL code. For example, assuming there is a text widget named Var1 that has input value "Hello", I would want to ...
- 18662 Views
- 6 replies
- 3 kudos
- 3 kudos
It seems that only way to use parameters in sql code block is to use dbutils.widget and you cannot change those parameters without removing widget and setting it up again in code
- 3 kudos
- 738 Views
- 0 replies
- 1 kudos
Databricks Policies and Bundles Inheritance: Let Policies Rule Your DABS
Just the policy_id can specify the entire cluster configuration. Yes, we can inherit default and fixed values from policies. Updating runtime version for 100s of jobs, for example, is much easier this way. Read more: - https://databrickster.med...
- 738 Views
- 0 replies
- 1 kudos
- 1614 Views
- 1 replies
- 3 kudos
Relationship in databricks Genie
Now you can define relations also directly in Genie. It includes options like “Many to One”, “One to Many”, “One to One”, “Many to Many”. Read more: - https://databrickster.medium.com/relationship-in-databricks-genie-f8bf59a9b578 - https://www.su...
- 1614 Views
- 1 replies
- 3 kudos
- 3 kudos
Thank you for sharing with the Community, @Hubert-Dudek!
- 3 kudos
- 2916 Views
- 1 replies
- 3 kudos
Real-Time Mode in Apache Spark Structured Streaming
Real-Time Mode in Spark StreamingApache Spark™ Structured Streaming has been the backbone of mission-critical pipelines for years — from ETL to near real-time analytics and machine learning.Now, Databricks has introduced something game-changing: Real...
- 2916 Views
- 1 replies
- 3 kudos
- 3 kudos
And now in Public Preview! Thank you for writing this up, @Yogesh_Verma_.
- 3 kudos
- 2402 Views
- 1 replies
- 4 kudos
How Databricks Helped Me See Data Engineering Differently
Over the years working as a data engineer, I’ve started to see my role very differently. In the beginning, most of my focus was on building pipelines—extracting, transforming, and loading data so it could land in the right place. Pipelines were the g...
- 2402 Views
- 1 replies
- 4 kudos
- 4 kudos
@Brahmareddy thanks for this! .Think you've nailed it on the head there. If the stakeholders trust the data and there's integrity, governance, and a single source of truth, you've got a recipe for a great product! Love this take @Brahmareddy . Really...
- 4 kudos
- 1806 Views
- 2 replies
- 6 kudos
(Teaser CHALLENGE - Community Data Pull) - Upcoming Challenge For the Community
Hey everybody, I've been dying to share this with the community. Over the last few weeks, I've been thinking about how I can do a Data Pull from the Community to highlight some of the cool stuff we all do! 拾. Below is a snippet of visual from the Dat...
- 1806 Views
- 2 replies
- 6 kudos
- 6 kudos
I'll aim to have the data for the challenge sorted and ready for next week! . I want to strip out some of the columns and figure out where is best to host the data . Potentially I could have it on the Databricks Marketplace or Github.All the best,BS
- 6 kudos
- 14562 Views
- 1 replies
- 3 kudos
9 Powerful 🚀 Spark Optimization Techniques in Databricks (With Real Examples)
IntroductionOne of our ETL pipelines used to take 10 hours to complete. After tuning and scaling in Databricks, it finished in just about 1 hour — a 90% reduction in runtime.That’s the power of Spark tuning.Databricks, built on Apache Spark, is a po...
- 14562 Views
- 1 replies
- 3 kudos
- 3 kudos
This is a fantastic breakdown of Spark optimization techniques, @savlahanish27!Definitely helpful for anyone working on performance tuning in Databricks.
- 3 kudos
- 2309 Views
- 0 replies
- 4 kudos
Semantic Modelling and Data Products using Agent
Hey Community!We have built something cool for Data Engineers in Databricks!Raw Files -> Semantic Model -> Data Products without writing ETL/ELT Code.Demo/ Guide - https://youtu.be/wjQYXrBwA-oNotebook - https://github.com/Intugle/data-tools/blob/main...
- 2309 Views
- 0 replies
- 4 kudos
- 5965 Views
- 2 replies
- 7 kudos
Data Quality at Scale: My Experience Using Databricks and AWS
Over the past few years working as a data engineer, I’ve seen how quickly companies are moving their platforms to Databricks and AWS. The flexibility and scale these platforms provide are amazing, but one challenge always comes up again and again: ho...
- 5965 Views
- 2 replies
- 7 kudos
- 7 kudos
Hi @Brahmareddy very good insights , I can summarize this as follows: Area Best Practice ExampleSchema ManagementDefine schemas in JSON/YAML, enforce with Delta LakeGovernanceUse Unity Catalog for access, lineage, and ownershipMonitoringSet up Lakeho...
- 7 kudos
- 12452 Views
- 4 replies
- 6 kudos
From Pilots to Production: Unlocking Enterprise AI Agents with Agent Bricks
Many enterprises today launch AI agents with high hopes but more often than not, those pilots never reach production. The culprit? Complexity, poor evaluations, ballooning costs, and governance gaps.Why do so many AI agent pilots never make it to pro...
- 12452 Views
- 4 replies
- 6 kudos
- 6 kudos
Agent Bricks looks solid for scaling AI, and I’ve seen platforms like Agentra.io also tackle the enterprise workflow side of this challenge.
- 6 kudos
- 15195 Views
- 4 replies
- 1 kudos
Databricks Academy Labs Coupon Instructions
To see all Databricks training and enablement offerings, please visit our Learning Library and Certifications Catalog. To use your Databricks Academy Labs coupons, please - Head to Databricks Academy Across the top navigation, select Subscriptions C...
- 15195 Views
- 4 replies
- 1 kudos
- 2176 Views
- 1 replies
- 4 kudos
Accelerating Data Migration and Data Engineering with AI: The Future of Databricks Adoption
In today’s fast-evolving digital landscape, organizations are under immense pressure to modernize their data infrastructure for better scalability, agility, and advanced analytics. One of the most powerful shifts in recent times has been the migratio...
- 2176 Views
- 1 replies
- 4 kudos
- 4 kudos
Great summary, @JatinArora! Clear and highlights the tangible benefits perfectly.
- 4 kudos
- 2439 Views
- 2 replies
- 4 kudos
What Does “Full Stack Development” Mean in the World of Databricks
Hi All, Many teams using Databricks today are referring to their work as “full stack development,” which can be a bit confusing at first. In the Databricks context, this doesn't mean a new framework — it simply means handling everything from raw data...
- 2439 Views
- 2 replies
- 4 kudos
- 4 kudos
Absolutely agree — what’s called “full stack” in Databricks truly means managing the complete data lifecycle on a unified platform,. Teams today aren’t just doing ETL; they’re orchestrating everything from real-time ingestion (Autoloader), scalable s...
- 4 kudos
- 2257 Views
- 1 replies
- 3 kudos
corrupted delta logs
ERROR: DeltaVersionsNotContiguousException: Versions (0, 2) are not contiguous. This can happen when files have been manually removed from the Delta log. Please contact Databricks support to repair the table.Cause of the error: You are getting this e...
- 2257 Views
- 1 replies
- 3 kudos
- 3 kudos
Thanks for sharing @Khaja_Zaffer!
- 3 kudos
- 8433 Views
- 4 replies
- 2 kudos
Demo Deploy a Databricks Asset Bundle with Azure DevOps Pipelines
I recently wrote a blog post showing how to deploy a minimal Databricks job with notebook task using Databricks Asset Bundles and Azure DevOps pipelines. The code is available in my public GitHub repo. Thought I'd share it here!ObjectivesIn this post...
- 8433 Views
- 4 replies
- 2 kudos
- 2 kudos
HiGreat article.2 questionsHow does the deploy bundle script in the pipleine know where to pick up the bundel. Should there be a path. Mine is on my local machine becaust that where I first run it from the cli. Also can a bundle deploy resources that...
- 2 kudos
| User | Count |
|---|---|
| 85 | |
| 75 | |
| 72 | |
| 60 | |
| 43 |
