Grow Your Skills and Earn Rewards! Mark your calendar: January 09 – January 30, 2026 Join us for a three-week event dedicated to learning, upskilling, and advancing your career in data engineering, analytics, machine learning, and generative AI. ...
- 124095 Views
- 389 replies
- 137 kudos
12-09-2025
June 15–18, 2026 Are you building the future with data and AI? Then this is your moment. The Call for Proposals for Data + AI Summit 2026 is officially open, and we want to hear from builders, practitioners, and innovators across the data and AI com...
- 7842 Views
- 4 replies
- 6 kudos
12-15-2025
How to Register & Prepare If you're interested in advancing your skills with Databricks through a Specialist Session, here's a clear guide on how to register and what free courses you can take to prepare effectively. How to Begin Your Learning Path S...
- 3996 Views
- 2 replies
- 9 kudos
10-02-2025
Strengthening cybersecurity isn’t a one-time project — it’s an ongoing process of adapting to new and complex threats. The focus is on modernizing your data infrastructure and analytics capabilities to make security operations smarter, faster, and mo...
- 980 Views
- 1 replies
- 2 kudos
2 weeks ago
2025 wasn’t just another year on the Databricks Community.It was a year of showing up, helping each other, and building trust. Watch the 2025 Community Wrap-Up This video is a small reflection of that journey.It highlights the key contributors of 2...
- 1435 Views
- 13 replies
- 20 kudos
2 weeks ago
Another strong week of engagement, learning, and problem-solving across the community.Let’s take a look at the highlights Standout Contributors These members stepped in with sharp insights, thoughtful follow-ups, and the kind of curiosity that ke...
- 282 Views
- 0 replies
- 0 kudos
Tuesday
Community Activity
- 48 Views
- 0 replies
- 0 kudos
Invitation to the Databricks Developer Community on Medium
Hi All,We are starting a new Databricks Developer Community publication on Medium, and this is a warm invitation to every Databricks professional, developer, and aspirant who believes in learning by sharing.We are doing this because Databricks learni...
- 48 Views
- 0 replies
- 0 kudos
- 60 Views
- 2 replies
- 1 kudos
Resolved! The Lakeflow connect Gateway setup, do we need to install the agent on-prem?
The Lakeflow connect Gateway setup to connect on-prem SQL serverPlease provide the steps the setup the gateway agent on the on-prem? Where to download this agent? What are the firewall rules for outbound looks like. Kind regards,Asha
- 60 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @Ashash12 ,You need to have proper network connectivity to your on premise SQL Server. As they stated in the docs - connector supports SQL Server on-premises using Azure ExpressRoute and AWS Direct Connect networkinghttps://docs.databricks.com/aws...
- 1 kudos
- 20 Views
- 0 replies
- 0 kudos
Agent Skills
Do you know that it is possible to extend the Assistant with agent skills? It is really straightforward and allows you, in fact, to extend the functionality of databricks. You can create templates for an assistant - I experimented with a template to ...
- 20 Views
- 0 replies
- 0 kudos
- 1065 Views
- 4 replies
- 1 kudos
Resolved! Job fails on clusters only with library dependency
Hello!I have following problem: All my job runs fail when the job uses a library. Even the most basic job (print a string) and the most basic library package (no secondary dependencies, the script does not even import/use the library) fails with `Fai...
- 1065 Views
- 4 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi, can you please share detailed soluton, did you include any config in databricks.yml? I'm also having same issue.
- 1 kudos
- 105 Views
- 3 replies
- 0 kudos
Can't Access Agents
Hi there, I'm very new to Databricks and I wanted to try out Agents. At first, I thought I couldn't access Agents because I was on a Trial account, so I added my CC to the subscription. Unfortunately, I still can't access the feature, not to mention ...
- 105 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi Szymon, thanks for a prompt reply!I followed your suggestions (AWS version) but sadly it seems my account is not on the allowlist for Mosaic Beta.I messaged Databricks support via email, but it seems I also don't have access to the Help Center eit...
- 0 kudos
- 118 Views
- 2 replies
- 1 kudos
Best practices for 3-layer access control in Databricks
Identity and access management model for Databricks and want to implement a clear 3-layer authorization approach:Account level: Account RBAC roles (account admin, metastore admin, etc.)Workspace level: Workspace roles/entitlements + workspace ACLs (c...
- 118 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Thank you for the detailed explanation and for sharing the reference blogs, I may follow up once we complete our initial design draft.
- 1 kudos
- 95 Views
- 1 replies
- 0 kudos
Bug Report: SDP (DLT) with autoloader not passing through pipe delimiter/separator
I am noticing a difference between using autoloader in an interactive notebook vs using it in a Spark Declarative Pipeline (DLT Pipeline). This issue seems to be very similar to this other unanswered post from a few years ago. Bug report: the delimit...
- 95 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @ChrisLawford_n1, can you try doing a full refresh of this table? The old schema may still be in the schema location.
- 0 kudos
- 45 Views
- 0 replies
- 0 kudos
Hive metastore CANNOT access storage
I'm new to Azure Databricks and I'm facing an issue when trying to create a schema or table that points to my Azure Storage account. I keep getting this error:```[EXTERNAL_METASTORE_CLIENT_ERROR.OPERATION_FAILED] Client operation failed: org.apache.h...
- 45 Views
- 0 replies
- 0 kudos
- 60 Views
- 0 replies
- 0 kudos
Great Expectations with Fabric and Databricks-Building Bulletproof Data Pipelines
In this article, I’ll walk you through transforming a basic PySpark notebook into a production-ready data pipeline with comprehensive quality checks using Great Expectations patterns in Microsoft Fabric and Databricks. We’ll start simple and progress...
- 60 Views
- 0 replies
- 0 kudos
- 766 Views
- 4 replies
- 5 kudos
Resolved! switching to Databrick from Ab Initio (an old ETL software)- NEED ADVICE
All courses in market and on youtube as per my knowledge for databrick is outdated as those courses are for community edition. there is no new course for free edition of databrick. i am a working profession and i do not get much time. do you guys kno...
- 766 Views
- 4 replies
- 5 kudos
Latest Reply
- 5 kudos
@pop_smoke What were the arguments that swayed your organization to swtich to Databricks from Ab Initiio?Pure cost basis?
- 5 kudos
- 69 Views
- 0 replies
- 1 kudos
Databricks Assistant
Databricks Assistant can also be used in databricks documentation without login to #databricks. Read and watch databricks news on:https://databrickster.medium.com/databricks-news-2026-week-2-5-january-2026-to-11-january-2026-0bfc6c592051
- 69 Views
- 0 replies
- 1 kudos
- 49 Views
- 1 replies
- 0 kudos
al purpose databricks cluster disappear
Hi, i can see sometime the cluster get disappeared even though it was created some time back using cluster pipeline,what could be the reason to disappear.we can recreate cluster but wanted to know the reason why this cluster get disappeared.thanks!Ve...
- 49 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Ved88 ,30 days after a compute is terminated, it is permanently deleted. To keep an all-purpose compute configuration after a compute has been terminated for more than 30 days, an administrator can pin the compute. Up to 100 compute resources can...
- 0 kudos
- 52 Views
- 0 replies
- 0 kudos
Publish a Technical Blog
Sir/Madam,I hope you are doing well. I would like to contribute and publish a technical blog on the Databricks Community portal. However, I am not sure about the process and appropriate platform. I would appreciate your guidance on where to start, th...
- 52 Views
- 0 replies
- 0 kudos
- 66 Views
- 2 replies
- 0 kudos
Same Delta Table, Different Behavior: Dev vs Prod Workspace in Databricks
I recently ran into an interesting Databricks behavior while implementing a row-count comparison using Delta Time Travel (VERSION AS OF).Platform: AzureScenario:Same Unity CatalogSame fully qualified tableSame table ID, location, and Delta formatYet ...
- 66 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
This could happen if the Delta table in production is a streaming table, while the Delta table in development is a regular Delta table. Time travel doesn’t work well with streaming tables and materialized views.
- 0 kudos
- 83 Views
- 0 replies
- 0 kudos
Unity Catalog Schema management
From time to time i read articles such as here which suggest to use a unity catalog schema management tool. All table schema changes should be applied via this tool.Usually SPs (or users) have the "Modify" Permission on tables. This allows to them t...
- 83 Views
- 0 replies
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2247 | |
| 1227 | |
| 1088 | |
| 1037 | |
| 950 |
Latest from our Blog
10 things you no longer need to think about (but if you are, here's how to fix it)
Databricks just released a button to fix all legacy issues in your workspace. Actually, they didn't. But the fact that you felt a momentary surge of hope—or a cynical eye-roll—proves exactly where you...
From Traces to Evaluation: An Autonomous MLflow Agent Harness for AI AgentOps
The Gap in Agent Development Databricks has made it straightforward to deploy AI agents—Model Serving endpoints, automatic MLflow tracing, Unity Catalog integration. But there's a gap between "deploye...
Combining Agent Bricks and ai_parse_document for Smarter Document Workflows
Building a Research Paper Curator for Knowledge Assistants Document-heavy workflows such as research analysis, contract review, and support ticket processing, are a natural fit for AI agents. Building...
Declarative Pipelines meets foreachBatch: Custom Streaming for Advanced Pipelines
If this is the first time you’re hearing about Sinks in Lakeflow Declarative Pipelines (LDP), we highly recommend reading Introducing the SDP Sink API, which explores the recently launched Sinks API, ...
