Joining a regional user group is a great way to connect with data and AI professionals near you. These groups bring together practitioners to learn, network, and share real-world experiences — all within your local context. To join a group: Select y...
- 1271 Views
- 10 replies
- 7 kudos
2 weeks ago
Unity Catalog (UC) managed tables combine strong governance with seamless interoperability across tools. Since the data sits in the customer-owned cloud storage, organizations retain full control over its physical location, while benefiting from Data...
- 956 Views
- 0 replies
- 5 kudos
2 weeks ago
We’re excited to invite you to our MONTHLY Databricks Get Started Days, a half-day virtual learning experience designed to jumpstart your journey with Databricks. This global event is tailored to equip you with essential data engineering and analysi...
- 13995 Views
- 14 replies
- 5 kudos
04-03-2025
We’re excited to launch the North East UK User Group—a dedicated space for data enthusiasts, engineers, scientists, and professionals from across Newcastle, Sunderland, Durham, Teesside, and beyond to connect, collaborate, and grow together in the Da...
- 1668 Views
- 1 replies
- 2 kudos
3 weeks ago
Community Activity
- 131 Views
- 2 replies
- 0 kudos
Databricks serverless SQL cost-effective.. when?
Hey.I read at one-point that there would be a cheaper, less-performant version of the Databricks serverless SQL DW but can't find any information on this. Was I dreaming or will it soon be possible with a more cost-effective option for less-intensive...
- 131 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @jar ,It’s possible you’re referring to the performance mode for serverless compute. However, at the moment it doesn’t apply to DB SQL Warehouse:The first mode, which is in General Availability, is called performance-optimized mode. This mode is d...
- 0 kudos
- 7 Views
- 0 replies
- 0 kudos
Is it possible to run Databricks Genie entirely within a private network ?
We're considering using Databricks Genie in a secure environment. Is there a way to run Genie fully on an internal network without requiring outbound internet access?
- 7 Views
- 0 replies
- 0 kudos
- 161 Views
- 7 replies
- 1 kudos
Problem activating File Events for External Location / ADLS V2
Hi allI've followed the book for creating external location for Azure Data Lake Storage (ADLS V2) using account connectorI've granted all required permissions to the connector:I've created a "stock" container on that above mentioned "devtyremeshare" ...
- 161 Views
- 7 replies
- 1 kudos
Latest Reply
- 1 kudos
I dont know how can we connect on teams to resolve the issue. is it possible? I am not working for msft or databricks currently. so asking you to join for a call is against the community guidance? I am not sure of this.
- 1 kudos
- 46 Views
- 1 replies
- 0 kudos
Test - Nick Iverson
12345
- 46 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi Nick, is there something you need?
- 0 kudos
- 88 Views
- 3 replies
- 1 kudos
Databricks Apps with Pyodbc Microsoft SQL Driver
I'm building an app that interfaces with an Azure SQL Database. I need to use Entra auth with a service principal, which is why I'm using the Microsoft ODBC driver. However, this works fine on my local, but I can't figure out how to get the ODBC driv...
- 88 Views
- 3 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi @bcodernet ,Each Databricks app can include dependencies for Python, Node.js, or both. You define these dependencies in language-specific files:Use a requirements.txt file to specify additional Python packages.Use a package.json file to specify No...
- 1 kudos
- 41 Views
- 0 replies
- 0 kudos
databricks bundle install: Error: Maximum file size of 524288000 exceeded
I have a job that's running some ML classification models. This uses PyTorch 2.5.0. I've configured the project with that dependency. I can deploy my job to our dev system from my laptop and all goes well. When I run this off our CI/CD server, for so...
- 41 Views
- 0 replies
- 0 kudos
- 56 Views
- 2 replies
- 1 kudos
[Feedback Requested] Data Domains, and how your organization uses them
Hi everyone, As part of our plans to improve the Discover Page, one of the next items we are looking into is building out Domains. 1. How have are your organizations using Data Domains today? 2. Can you share an example of your domain (and subdomain...
- 56 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Hello @PeterWang Good day!! Good to see that databricks tries to improve everyday and its always day1 at databricks. I think Abac would be ideal if domains are used. again its upto you tho.
- 1 kudos
- 26070 Views
- 17 replies
- 26 kudos
Databricks Announces the Industry’s First Generative AI Engineer Learning Pathway and Certification
Today, we are announcing the industry's first Generative AI Engineer learning pathway and certification to help ensure that data and AI practitioners have the resources to be successful with generative AI. At Databricks, we recognize that generative ...
- 26070 Views
- 17 replies
- 26 kudos
Latest Reply
- 26 kudos
Thanks for the post Are the course materials online somewhere? I've been following along with the vids and retyping out most of the code from the screen (which is a good way to learn it in depth but I've reached the section that uses Graphviz in "...
- 26 kudos
- 438 Views
- 10 replies
- 3 kudos
AI/BI dashboard integration in HTML page
Hey Team,Have below queries :I want to test AI/BI dashboard embeding and want to embed in an html page I have created, but dashboard is not getting loaded can you please help in understanding process.And also I want user to have access even if they ...
- 438 Views
- 10 replies
- 3 kudos
- 55 Views
- 1 replies
- 0 kudos
Data engineer associate exam was suspended, need help
Hi @Cert-Team I would require your help on the below support ticket.Request ID: #00699279I faced some interruptions and challenges while attempting my certification. My exam got suspended just because I leaned closer to the laptop screen to clearly r...
- 55 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@Bhargav14 follow the steps in this thread: https://community.databricks.com/t5/certifications/suspended-exam-don-t-worry-our-support-team-can-help/td-p/44469 Let me know how you get on. All the best,BS
- 0 kudos
- 46 Views
- 0 replies
- 0 kudos
Can't Access Labs
I have a new subscription to the labs and am enrolled in a labs enabled course (Data Ingestion with Lakeflow Connect). Nowhere can I find a link to access the lab material. The course looks identical to the free version. The screens in the "Accessing...
- 46 Views
- 0 replies
- 0 kudos
- 157 Views
- 2 replies
- 0 kudos
Converting Existing Streaming Job to Delta Live Tables with Historical Backfill
Description:I’m migrating a two-stage streaming job into Delta Live Tables (DLT):Bronze: read from Pub/Sub → write to Bronze tableSilver: use create_auto_cdc_flow on Bronze → upsert into Silver tableNew data works perfectly, but I now need to backfil...
- 157 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @yazz ,I’m wondering if you could use a similar approach to the one in the article below. So, just backfill your bronze table first. Then, the downstream silver and gold layers will pick up the new data from the bronze layer.In that approach you ...
- 0 kudos
- 308 Views
- 5 replies
- 12 kudos
Databricks Data Challenges for Upskilling
Hi all, I'm currently looking at upskilling in Databricks. I'd like to focus on becoming great at solving a variety of problems. I come with a background in the Alteryx community and they have something called Weekly Challenges: https://community.alt...
- 308 Views
- 5 replies
- 12 kudos
Latest Reply
- 12 kudos
@Advika let me know if I can be of any help with building this at all. To get maximum engagement from the community, it'll be great if there can be badges (these show on the community profile) associated to completing these challenges. I.e. first 1, ...
- 12 kudos
- 406 Views
- 2 replies
- 1 kudos
Community (Legacy) Edition Question
Hello,I have been teaching my Data Warehousing class using the Databricks Community Edition. With the change to the Databricks Free Edition, there are many aspects of my Community Edition notebooks that no longer work in the free Edition. Is there ...
- 406 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Hello @OU_Professor! Access to existing Community Edition accounts will remain available for the rest of the year. However, please note that new users attempting to sign up for Community Edition are now redirected to the Free Edition instead.
- 1 kudos
- 765 Views
- 3 replies
- 0 kudos
Enable automatic identity management in Azure Databricks
We have Databricks account admin access but not able to see the option from Databricks admin console to enable automatic identity management.Using the Previews page wanted to enable and fallowed below steps:1. As an account admin, log in to the accou...
- 765 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
After raising Databrick ticket, today I am able to see the Automatic Identity Management public preview option
- 0 kudos
Announcements
Latest from our Blog
Databricks for Identity Systems - Part 5 (Application Assignments)
In the previous section, we discovered how to pull Applications and Application Groups into Databricks. I briefly mentioned that we generally assign users to groups and then assign groups to applicati...
Databricks for Identity Systems - Part 4 (Applications)
Now that we have our Users, Groups, and Group Members exported to Databricks; it's time us to grab the applications that these users and groups are assigned to. Much like the previous walkthroughs, we...
How to scope a Unity Catalog (UC) Workflow Migration using System Tables
Pre-Requisites: Before diving into the workflow scoping process, it is essential to install UCX and run the assessment. The assessment will provide the initial scope of workflows in the workspace. In ...
Secure Storage Connectivity to Both Classic and Serverless Compute Using Terraform in Azure
Data security is the topmost priority and critical requirement for any organization in today's digital world. Data storage security refers to the measures and protocols implemented to protect data sto...
