- 1880 Views
- 4 replies
- 2 kudos
Demo Deploy a Databricks Asset Bundle with Azure DevOps Pipelines
I recently wrote a blog post showing how to deploy a minimal Databricks job with notebook task using Databricks Asset Bundles and Azure DevOps pipelines. The code is available in my public GitHub repo. Thought I'd share it here!ObjectivesIn this post...
- 1880 Views
- 4 replies
- 2 kudos
- 2 kudos
HiGreat article.2 questionsHow does the deploy bundle script in the pipleine know where to pick up the bundel. Should there be a path. Mine is on my local machine becaust that where I first run it from the cli. Also can a bundle deploy resources that...
- 2 kudos
- 2342 Views
- 1 replies
- 1 kudos
Installing CrowdStrike Falcon Sensor on Databricks Workers
Greetings,Does anyone here have experience deploying the CrowdStrike Falcon sensor on Databricks worker instances? For context, the cluster is deployed in AWS and we use a Databricks Ubuntu 20.04 AMI. Databricks allows adding a bootstrap/startup scri...
- 2342 Views
- 1 replies
- 1 kudos
- 1 kudos
I have the same doubt!
- 1 kudos
- 115 Views
- 0 replies
- 3 kudos
Accelerating Data Migration and Data Engineering with AI: The Future of Databricks Adoption
In today’s fast-evolving digital landscape, organizations are under immense pressure to modernize their data infrastructure for better scalability, agility, and advanced analytics. One of the most powerful shifts in recent times has been the migratio...
- 115 Views
- 0 replies
- 3 kudos
- 2214 Views
- 6 replies
- 1 kudos
Databricks App Availability
Hi there,I recently came across this post about databricks apps that says it available for public previewhttps://www.databricks.com/blog/introducing-databricks-appsHowever, when I go to previews in the workspace, I don't see an option to enable it, i...
- 2214 Views
- 6 replies
- 1 kudos
- 1 kudos
Yea same...I got the same warning "Requested region australiasoutheast in cloud azure is not supported."Has there been any further updates?
- 1 kudos
- 3894 Views
- 3 replies
- 6 kudos
From Pilots to Production: Unlocking Enterprise AI Agents with Agent Bricks
Many enterprises today launch AI agents with high hopes but more often than not, those pilots never reach production. The culprit? Complexity, poor evaluations, ballooning costs, and governance gaps.Why do so many AI agent pilots never make it to pro...
- 3894 Views
- 3 replies
- 6 kudos
- 6 kudos
Moving from pilots to full-scale production requires a solid foundation of enterprise AI software that can adapt to complex workflows and scale securely. With Agentra, enterprises can leverage Agent Bricks to build customizable AI agents that integra...
- 6 kudos
- 310 Views
- 3 replies
- 1 kudos
Databricks Apps Crashes Unexpectedly Without Showing any Logs
Hi All,I coded up a Databricks app using fastapi which seems to be crashing on deployment. Databricks throws the error that "app crashed at startup" but the logging page is empty. So I do not know what went wrong. Any ideas on how to debug the proble...
- 310 Views
- 3 replies
- 1 kudos
- 1 kudos
I want to update that I have been able to solve the problem and get the app running. The problem was related to trying to import from anoter folder instead of downloading the package using its wheel file. Just for the record, had already tried the l...
- 1 kudos
- 1872 Views
- 1 replies
- 6 kudos
Connecting VS Code and GitHub Copilot to the Databricks Managed MCP Server
Recently, Databricks released a preview version of the Managed MCP Server. Upon seeing this, I immediately wanted to integrate Databricks Genie with VS Code and GitHub Copilot agent mode. Below, I will briefly share the setup process:Step 1: Prepare ...
- 1872 Views
- 1 replies
- 6 kudos
- 6 kudos
Thanks for the example this is something I am looking for
- 6 kudos
- 279 Views
- 2 replies
- 3 kudos
Getting Started with Notebooks in Databricks
Databricks notebooks are a powerful tool for data scientists and engineers to collaborate, explore data, and build machine learning models. This guide will help you get started with creating and using notebooks in Databricks.Why Use Databricks Notebo...
- 279 Views
- 2 replies
- 3 kudos
- 3 kudos
Thanks for sharing @bhanu_gautam. This will surely help beginners get started with Databricks Notebooks.
- 3 kudos
- 421 Views
- 4 replies
- 4 kudos
Resolved! Unity Catalog Migration Strategy
Zero-Downtime Unity Catalog Migration for 500TB Data LakeJust completed migrating 500TB to Unity Catalog without a single minute of downtime. Here's how:The Challenge500 TB across 12,000 tables200+ concurrent usersZero tolerance for downtimeMixed Hiv...
- 421 Views
- 4 replies
- 4 kudos
- 4 kudos
Thanks, @Khaja_Zaffer and @BS_THE_ANALYST!@Khaja_Zaffer:The toolkit has 5 main components:Pre-migration analyzer Compatibility scoringDrift monitor Real time consistency checksPermission migrator: Automated ACL copyingQuery rewriter: Hive→UC SQL con...
- 4 kudos
- 10303 Views
- 10 replies
- 29 kudos
Zero to Hero - Data Analysis Certification
Hey everyone, Today I passed my Data Analyst Associate exam! 拾I'd like to say thanks to the Databricks community. The questions that people ask and interactions I've been a part of have been invaluable. Starting tomorrow, I'll be moving onto my data ...
- 10303 Views
- 10 replies
- 29 kudos
- 29 kudos
@bhanu_gautam I'm glad the guide resonates with you . Was there anything you'd add to it or change? .Congrats on passing the certification by the way 拾!! Have you got another certification or learning on the horizon now that you've passed this one? ....
- 29 kudos
- 137 Views
- 0 replies
- 3 kudos
GetRunbook_Failed :: Bootstrap timeout - cluster failure during startup
After creating a new workspace, if you come across Failed to get instance bootstrap steps from the Databricks Control Plane. Please check that instances have connectivity to the Databricks Control Plane. Instance bootstrap inferred timeout reason: Ge...
- 137 Views
- 0 replies
- 3 kudos
- 1141 Views
- 17 replies
- 62 kudos
Level Up Your Databricks Game - Episode 1: Widgets
Hi all,TheOC here with my first of (hopefully!) many blogs on the Databricks Community. I'm hoping in this series to share quick, practical tips to help you get the most out of Databricks. Today's topic is: Widgets.If you're anything like me, you've ...
- 1141 Views
- 17 replies
- 62 kudos
- 62 kudos
hey @Rishabh_Tiwari,Thank you!Watch this space for the next instalment
- 62 kudos
- 613 Views
- 4 replies
- 12 kudos
[Blog] Building a Scalable Telco CDR Processing Pipeline with Databricks Delta Live Tables - Part 1
Hey everyone! I wanted to share what I'm working with daily in the Databricks ecosystem and how amazing it is that we can achieve everything within one platform!Just published a deep dive on building a Telco CDR Processing Pipeline using: Delta Live ...
- 613 Views
- 4 replies
- 12 kudos
- 12 kudos
@Pat here I was, cup of tea in hand, ready and eager to read the blog, only then did I discover I was redirected to another place that actually hosts the full blog .Personally, I think it'd be nice for it to be on here, especially if it's dedicated t...
- 12 kudos
- 174 Views
- 0 replies
- 1 kudos
Technical Deep Dive
Bloom Filters + Zonemaps: The Ultimate Query Optimization ComboAfter my zonemap post last week got great feedback, several of you asked about Bloom filter integration. Here's the complete implementation!Why Bloom Filters Changed EverythingZonemaps ar...
- 174 Views
- 0 replies
- 1 kudos
- 800 Views
- 12 replies
- 7 kudos
Resolved! 🚀 DataFrame Caching on Delta Tables - What if underlying data is updated?
Just published new video on Databricks Performance Series to try to clearly explain how DataFrame caching over Delta Tables behaves when updates on underlying table are performed. I came across this use case in my recent project and struggled a littl...
- 800 Views
- 12 replies
- 7 kudos
- 7 kudos
Source Code with samples available at https://github.com/CafeConData/Spark-Caching-on-Delta-Tables
- 7 kudos
