Grow Your Skills and Earn Rewards! Mark your calendar: January 09 – January 30, 2026 Join us for a three-week event dedicated to learning, upskilling, and advancing your career in data engineering, analytics, machine learning, and generative AI. ...
- 121927 Views
- 382 replies
- 136 kudos
12-09-2025
June 15–18, 2026 Are you building the future with data and AI? Then this is your moment. The Call for Proposals for Data + AI Summit 2026 is officially open, and we want to hear from builders, practitioners, and innovators across the data and AI com...
- 7753 Views
- 4 replies
- 6 kudos
12-15-2025
How to Register & Prepare If you're interested in advancing your skills with Databricks through a Specialist Session, here's a clear guide on how to register and what free courses you can take to prepare effectively. How to Begin Your Learning Path S...
- 3991 Views
- 2 replies
- 9 kudos
10-02-2025
Strengthening cybersecurity isn’t a one-time project — it’s an ongoing process of adapting to new and complex threats. The focus is on modernizing your data infrastructure and analytics capabilities to make security operations smarter, faster, and mo...
- 950 Views
- 1 replies
- 2 kudos
2 weeks ago
2025 wasn’t just another year on the Databricks Community.It was a year of showing up, helping each other, and building trust. Watch the 2025 Community Wrap-Up This video is a small reflection of that journey.It highlights the key contributors of 2...
- 1412 Views
- 13 replies
- 20 kudos
2 weeks ago
Another strong week of engagement, learning, and problem-solving across the community.Let’s take a look at the highlights Standout Contributors These members stepped in with sharp insights, thoughtful follow-ups, and the kind of curiosity that ke...
- 267 Views
- 0 replies
- 0 kudos
Tuesday
Community Activity
- 81 Views
- 1 replies
- 0 kudos
Bug Report: SDP (DLT) with autoloader not passing through pipe delimiter/separator
I am noticing a difference between using autoloader in an interactive notebook vs using it in a Spark Declarative Pipeline (DLT Pipeline). This issue seems to be very similar to this other unanswered post from a few years ago. Bug report: the delimit...
- 81 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @ChrisLawford_n1, can you try doing a full refresh of this table? The old schema may still be in the schema location.
- 0 kudos
- 37 Views
- 1 replies
- 0 kudos
The Lakeflow connect Gateway setup, do we need to install the agent on-prem?
The Lakeflow connect Gateway setup to connect on-prem SQL serverPlease provide the steps the setup the gateway agent on the on-prem? Where to download this agent? What are the firewall rules for outbound looks like. Kind regards,Asha
- 37 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Ashash12 ,You need to have proper network connectivity to your on premise SQL Server. As they stated in the docs - connector supports SQL Server on-premises using Azure ExpressRoute and AWS Direct Connect networkinghttps://docs.databricks.com/aws...
- 0 kudos
- 35 Views
- 0 replies
- 0 kudos
Hive metastore CANNOT access storage
I'm new to Azure Databricks and I'm facing an issue when trying to create a schema or table that points to my Azure Storage account. I keep getting this error:```[EXTERNAL_METASTORE_CLIENT_ERROR.OPERATION_FAILED] Client operation failed: org.apache.h...
- 35 Views
- 0 replies
- 0 kudos
- 47 Views
- 0 replies
- 0 kudos
Great Expectations with Fabric and Databricks-Building Bulletproof Data Pipelines
In this article, I’ll walk you through transforming a basic PySpark notebook into a production-ready data pipeline with comprehensive quality checks using Great Expectations patterns in Microsoft Fabric and Databricks. We’ll start simple and progress...
- 47 Views
- 0 replies
- 0 kudos
- 758 Views
- 4 replies
- 5 kudos
Resolved! switching to Databrick from Ab Initio (an old ETL software)- NEED ADVICE
All courses in market and on youtube as per my knowledge for databrick is outdated as those courses are for community edition. there is no new course for free edition of databrick. i am a working profession and i do not get much time. do you guys kno...
- 758 Views
- 4 replies
- 5 kudos
Latest Reply
- 5 kudos
@pop_smoke What were the arguments that swayed your organization to swtich to Databricks from Ab Initiio?Pure cost basis?
- 5 kudos
- 53 Views
- 0 replies
- 1 kudos
Databricks Assistant
Databricks Assistant can also be used in databricks documentation without login to #databricks. Read and watch databricks news on:https://databrickster.medium.com/databricks-news-2026-week-2-5-january-2026-to-11-january-2026-0bfc6c592051
- 53 Views
- 0 replies
- 1 kudos
- 34 Views
- 1 replies
- 0 kudos
al purpose databricks cluster disappear
Hi, i can see sometime the cluster get disappeared even though it was created some time back using cluster pipeline,what could be the reason to disappear.we can recreate cluster but wanted to know the reason why this cluster get disappeared.thanks!Ve...
- 34 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Ved88 ,30 days after a compute is terminated, it is permanently deleted. To keep an all-purpose compute configuration after a compute has been terminated for more than 30 days, an administrator can pin the compute. Up to 100 compute resources can...
- 0 kudos
- 43 Views
- 0 replies
- 0 kudos
Publish a Technical Blog
Sir/Madam,I hope you are doing well. I would like to contribute and publish a technical blog on the Databricks Community portal. However, I am not sure about the process and appropriate platform. I would appreciate your guidance on where to start, th...
- 43 Views
- 0 replies
- 0 kudos
- 55 Views
- 2 replies
- 0 kudos
Same Delta Table, Different Behavior: Dev vs Prod Workspace in Databricks
I recently ran into an interesting Databricks behavior while implementing a row-count comparison using Delta Time Travel (VERSION AS OF).Platform: AzureScenario:Same Unity CatalogSame fully qualified tableSame table ID, location, and Delta formatYet ...
- 55 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
This could happen if the Delta table in production is a streaming table, while the Delta table in development is a regular Delta table. Time travel doesn’t work well with streaming tables and materialized views.
- 0 kudos
- 62 Views
- 0 replies
- 0 kudos
Unity Catalog Schema management
From time to time i read articles such as here which suggest to use a unity catalog schema management tool. All table schema changes should be applied via this tool.Usually SPs (or users) have the "Modify" Permission on tables. This allows to them t...
- 62 Views
- 0 replies
- 0 kudos
- 2376 Views
- 3 replies
- 4 kudos
Getting Databricks Generative AI Engineer Associate (and What I Learned)
I just earned my Databricks Certified Generative AI Engineer Associate Certification, and in this post, I’m sharing the key tips, resources, and personal insights that helped me succeed. My certiticate from DatabricksNavigation:PreludeAbout the certR...
- 2376 Views
- 3 replies
- 4 kudos
Latest Reply
- 4 kudos
Hi @devipriya Contrast and thanks' for shearing use full key tips, resources, and personal insights. it's really helpful for others...
- 4 kudos
- 39 Views
- 0 replies
- 0 kudos
Query: INR Payment Option & Certification Cost Considerations
Hello Team,I had a question regarding the payment options and overall cost for Databricks certifications, particularly from an India-based learner perspective.Is there currently an option to pay the certification fees in INR? If not, I wanted to high...
- 39 Views
- 0 replies
- 0 kudos
- 54 Views
- 0 replies
- 1 kudos
Security considerations and model customization options for AgentBricks AI Agents and LLM Judges
Hello Databricks Team,We are currently evaluating AgentBricks AI Agents (for example, Knowledge Assistant and Multi-Agent Supervisor) and would like to better understand the security and model customization aspects.Data security & privacyWhat data (u...
- 54 Views
- 0 replies
- 1 kudos
- 51 Views
- 0 replies
- 0 kudos
Automate Resource Creation using Databricks SDK for Python in VS Code
In this video article, I demonstrated how to automate resource creation in Azure Databricks using the Databricks SDK for Python. If you've ever found yourself manually creating catalogs, schemas, and volumes in the Databricks UI repeatedly, this vid...
- 51 Views
- 0 replies
- 0 kudos
- 117 Views
- 0 replies
- 0 kudos
Let's talk about Data Governance
As we move into 2026, the mandate for modern data and AI is clear. The industry has pivoted from evaluating isolated features to building foundational futures. Since the 2025 Data + AI Summit, I have consulted with numerous organizations across a ran...
- 117 Views
- 0 replies
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2247 | |
| 1227 | |
| 1087 | |
| 1036 | |
| 950 |
Latest from our Blog
10 things you no longer need to think about (but if you are, here's how to fix it)
Databricks just released a button to fix all legacy issues in your workspace. Actually, they didn't. But the fact that you felt a momentary surge of hope—or a cynical eye-roll—proves exactly where you...
From Traces to Evaluation: An Autonomous MLflow Agent Harness for AI AgentOps
The Gap in Agent Development Databricks has made it straightforward to deploy AI agents—Model Serving endpoints, automatic MLflow tracing, Unity Catalog integration. But there's a gap between "deploye...
Combining Agent Bricks and ai_parse_document for Smarter Document Workflows
Building a Research Paper Curator for Knowledge Assistants Document-heavy workflows such as research analysis, contract review, and support ticket processing, are a natural fit for AI agents. Building...
Declarative Pipelines meets foreachBatch: Custom Streaming for Advanced Pipelines
If this is the first time you’re hearing about Sinks in Lakeflow Declarative Pipelines (LDP), we highly recommend reading Introducing the SDP Sink API, which explores the recently launched Sinks API, ...
