Databricks Free Edition is the new home for personal learning and exploration on Databricks. It’s perpetually free and built on modern Databricks - the same Data Intelligence Platform used by professionals. Free Edition lets you learn professional da...
- 631 Views
- 0 replies
- 4 kudos
Monday
June 15–18, 2026 Are you building the future with data and AI? Then this is your moment. The Call for Proposals for Data + AI Summit 2026 is officially open, and we want to hear from builders, practitioners, and innovators across the data and AI com...
- 922 Views
- 2 replies
- 5 kudos
Monday
Your voice matters to us. We are planning the 2026 Data + AI Summit, and we’d love your input on what would make the experience even more valuable for you. Take a few minutes to share your feedback through our quick survey — your insights directly in...
- 468 Views
- 3 replies
- 4 kudos
Monday
How to Register & Prepare If you're interested in advancing your skills with Databricks through a Specialist Session, here's a clear guide on how to register and what free courses you can take to prepare effectively. How to Begin Your Learning Path S...
- 3125 Views
- 2 replies
- 9 kudos
10-02-2025
Governing the Full Data Lifecycle: From Ingestion to Insight with Unity Catalog Date: Saturday, December 13 Time: 12:00 PM – 3:00 PM Location: Bengaluru, Karnataka (Register to see address) Registration: Approval required Register Here! About Thi...
- 927 Views
- 1 replies
- 3 kudos
3 weeks ago
Event Details Date: 19 December 2025Time: 6:00 PM – 7:30 PM ISTFormat: Virtual (Online) Marking RSVP on this page will not register your interest for the event. Please use the button below. Unlimited seats — but registration is recommended to recei...
- 951 Views
- 0 replies
- 6 kudos
2 weeks ago
Our Champion program has always celebrated the customers who go above and beyond to engage, help others, and uplift the Community. Recently, we have seen remarkable participation from Bricksters as well—and their impact deserves recognition too. Begi...
- 1242 Views
- 6 replies
- 13 kudos
a month ago
We’re inching toward year-end, And the Community just dropped another week packed with sharp questions and standout wins.If you missed the action, here’s the pulse of everything that made the feed light up Week’s Power Players Spotlight on t...
- 414 Views
- 4 replies
- 4 kudos
a week ago
Community Activity
- 5 Views
- 0 replies
- 0 kudos
while loading data from dataframe to spark sql table using .saveAstable() option, not working.
hi , i am loading dataframe data into spark sql table using .saveastable() option.. scema is matching..but column names are diffirent in sql table. is it necessary to maintain the same column names in source and target ? how to handle it in real time...
- 5 Views
- 0 replies
- 0 kudos
- 107 Views
- 1 replies
- 0 kudos
how to complete connection to snowflake
Hi all,Simple test creating connection to my snowflake trial account was successful, and..Anyway:dbx > external data > connections > create connection > ... > [Test] successful but still see last 2 steps todo (5 access, 6 metadata) but don't see any ...
- 107 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @emanueol ,From you description it seems that you were able to set up connection. Not every step in the wizard has "next button". From example in step 4 (catalog basic) if you create a catalog then you will be automatically transfered to the next ...
- 0 kudos
- 16 Views
- 0 replies
- 0 kudos
Databricks Advent Calendar 2025 #21
Your stream can have a state, and now, with TransformWithStateInPandas, it’s easy to manage - you can handle things like initial state, deduplication, recovery, etc., with the 2025 improvements.
- 16 Views
- 0 replies
- 0 kudos
- 11209 Views
- 8 replies
- 0 kudos
Log files are not getting deleted automatically after logRetentionDuration internal
Hi team Log files are not getting deleted automatically after logRetentionDuration internal from delta log folder and after analysis , I see checkpoint files are not getting created after 10 commits . Below table properties using spark.sql( f""" ...
- 11209 Views
- 8 replies
- 0 kudos
- 101 Views
- 0 replies
- 0 kudos
Databricks Advent Calendar 2025 #20
As Unity Catalog becomes an enterprise catalog, bring-your-own lineage is one of my favorite features.
- 101 Views
- 0 replies
- 0 kudos
- 161 Views
- 1 replies
- 0 kudos
Queries from Serverless compute referring to older/deleted/vacuumed version of the delta tables.
Hi Team,I have a unity catalog based managed delta table which I am able to successfully query using the regular compute/cluster options.But when I try to query the same table using a Serverless/SQL Warehouse, they are referring to an older version /...
- 161 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @s_agarwal Please find below my findinsg for your query. Serverless uses cached Unity Catalog metadata Your UC metadata points to an old Delta version Regular clusters bypass this cache Fix by refreshing or forcing UC metadata rewrite
- 0 kudos
- 152 Views
- 1 replies
- 0 kudos
spark conf for serveless jobs
Hello Guys, I use serveless on databricks Azure, so i have build a decorator which instanciate a SparkSession. My job use autolaoder / kafka using mode availableNow. Someone Knows which spark conf is required beacause i want to add it ? Thanx import...
- 152 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @seefoods Please find below my findings for your case. You don’t need (and can’t meaningfully add) any Spark conf to enable availableNow on Databricks Serverless. Let me explain clearly, and then show what is safe to do in your decorator. availa...
- 0 kudos
- 11651 Views
- 18 replies
- 16 kudos
Free Certification voucher
looking for free certification voucher
- 11651 Views
- 18 replies
- 16 kudos
Latest Reply
- 16 kudos
Hi, can you please provide me free voucher for azure data bricks fundamental
- 16 kudos
- 43607 Views
- 48 replies
- 5 kudos
Resolved! Data Engineer Associate Certification in 2025 Voucher
Hi Community,I’m planning to take the Data Engineer Associate Certification in 2025 to advance my skills and career. Are there any ongoing promotions or opportunities to get a 50% discount voucher for the exam? It would be incredibly helpful if you c...
- 43607 Views
- 48 replies
- 5 kudos
- 303 Views
- 5 replies
- 0 kudos
Read Array of Arrays of Objects JSON file using Spark
Hi Databricks Community! This is my first post in this forum, so I hope you can forgive me if it's not according to the forum best practices After lots of searching, I decided to share the peculiar issue I'm running into in this community.I try to lo...
- 303 Views
- 5 replies
- 0 kudos
Latest Reply
- 0 kudos
I guess I was a bit over enthusiastic by accepting the answer.When I run the following on the single object array of arrays (as shown in the original post) I get a single row with column "value" and value null. from pyspark.sql import functions as F,...
- 0 kudos
- 147 Views
- 0 replies
- 0 kudos
Removing access to Lakehouse and only allowing Databricks One?
Hello, I am trying to set up a user group for business users in our Azure Databricks that will only be able to query data. It looks like Databricks One is the solution to use. So I followed the documentation and granted the user group Consumer Access...
- 147 Views
- 0 replies
- 0 kudos
- 127 Views
- 0 replies
- 0 kudos
Can I create a serverless budget policy via Python SDK on Azure Databricks?
Hi, I am trying to use the Databricks Python SDK (v0.74.0) to automate the creation of budget policies in our Databricks account. See the Python code below where I am trying to create a serverless budget policy. Note the error.When I click the "Diagn...
- 127 Views
- 0 replies
- 0 kudos
- 129 Views
- 0 replies
- 0 kudos
Databricks Advent Calendar 2025 #19
In 2025, Metrics Views are becoming the standard way to define business logic once and reuse it everywhere. Instead of repeating complex SQL, teams can work with clean, consistent metrics.
- 129 Views
- 0 replies
- 0 kudos
- 194 Views
- 1 replies
- 0 kudos
How do I grant access to find a table in Databricks, without giving access to query the table?
Hi!By default it seems users can only see tables and views in Unity Catalog that they have SELECT permission/privilege on. However, we would like to use Unity Catalog as a data catalog of tables we have. They wouldn't then be able to request access t...
- 194 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@excavator-matt you can grant BROWSE privilege on your catalog to a broad audience (for example, the “All account users” group). This lets users see object metadata (names, comments, lineage, search results, information_schema, etc.) in Catalog Explo...
- 0 kudos
- 177 Views
- 1 replies
- 0 kudos
scroll bar disappears on widgets in dashboards
Databricks newbie.I've created a dashboard that has several widgets to allow users to select multiple values from a drop-down list. When I first open the widget to select the values, there is a scroll bar on the right side of the box which allows me...
- 177 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello @RichC! You’re not missing any setting here. This is expected behavior. The scrollbar auto-hides after a couple of seconds, but it’s still active. If you start scrolling again (mouse wheel or trackpad), the scrollbar will reappear.
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2192 | |
| 1214 | |
| 1056 | |
| 950 | |
| 878 |
Latest from our Blog
[PARTNER BLOG] Beyond ADLS Limits: File Arrival Triggers for File Updates
Beyond ADLS Limitations: Making File Arrival Triggers Work for Existing File Updates Using a Flag File MechanismThe Flag File MechanismThe Root Problem: Triggers Only Work on “Create”, Not “Modify” Ev...
AI Newsletter Series (Part 1): Parsing Enterprise PDFs Into AI-Ready Data
Enterprise Account macro trends, strategy doc, account and project updates often end up in PDF format. Meanwhile, usage metrics and account-level signals—such as active users, DBU consumption, and use...
Fine-Grained Access Control in Delta Sharing: Dynamic Views vs Partition Filtering
The Challenge: Sharing Data While Maintaining Privacy Boundaries Imagine you're a global retail company with customer order data spanning multiple regions—North America, Europe, and Asia Pacific. You ...
Integration Testing for Lakeflow Jobs with Pytest and Databricks Connect
Introduction Databricks Lakeflow enables data teams to design and operate data pipelines at scale, where speed and reliability directly influence the time to market for insights. As pipeline complexit...
