Databricks Free Edition is the new home for personal learning and exploration on Databricks. It’s perpetually free and built on modern Databricks - the same Data Intelligence Platform used by professionals. Free Edition lets you learn professional da...
- 1470 Views
- 1 replies
- 4 kudos
2 weeks ago
June 15–18, 2026 Are you building the future with data and AI? Then this is your moment. The Call for Proposals for Data + AI Summit 2026 is officially open, and we want to hear from builders, practitioners, and innovators across the data and AI com...
- 1967 Views
- 4 replies
- 6 kudos
2 weeks ago
Your voice matters to us. We are planning the 2026 Data + AI Summit, and we’d love your input on what would make the experience even more valuable for you. Take a few minutes to share your feedback through our quick survey — your insights directly in...
- 862 Views
- 3 replies
- 5 kudos
2 weeks ago
How to Register & Prepare If you're interested in advancing your skills with Databricks through a Specialist Session, here's a clear guide on how to register and what free courses you can take to prepare effectively. How to Begin Your Learning Path S...
- 3373 Views
- 2 replies
- 9 kudos
10-02-2025
Our Champion program has always celebrated the customers who go above and beyond to engage, help others, and uplift the Community. Recently, we have seen remarkable participation from Bricksters as well—and their impact deserves recognition too. Begi...
- 1512 Views
- 7 replies
- 14 kudos
11-21-2025
Learning doesn’t pause, and neither does the impact this Community continues to create!Across threads and time zones, the knowledge kept moving. Catch up on the highlights Voices Shaping the Week Featuring the voices that brought clarity, ...
- 467 Views
- 1 replies
- 1 kudos
a week ago
Community Activity
- 18 Views
- 1 replies
- 0 kudos
ModuleNotFoundError: No module named 'MY-MODEL'
I'm currently trying to create a model serving end point around a model I've recently created. I'm trying to wrap my head around an error. The model is defined as below class MY-MODEL(mlflow.pyfunc.PythonModel): def load_context(self, context): ...
- 18 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
As an update got the model deployed and serving traffic. Now the issue is ModuleNotFoundError: No module named 'FILE-MY-MODEL-IMPORTS' I also did update those file locations to a volume in the same schema as the model, based on a response to another...
- 0 kudos
- 64 Views
- 2 replies
- 2 kudos
Resolved! Databricks Free Edition - Accessing files in S3
Hello,Attempting read/write files from s3 but got the error below. I am on the free edition (serverless by default). I'm using access_key and secret_key. Has anyone done this successfully? Thanks!Directly accessing the underlying Spark driver JVM us...
- 64 Views
- 2 replies
- 2 kudos
- 14 Views
- 0 replies
- 0 kudos
How to not make tab headers show when embedding dashboard
When embedding the AI BI dashboard, is there a way to not make the tabs show and instead use our own UI tab to navigate the tabs?Currently, there are two tab headers - one in the databricks dashboard and then another tab section in our embedding webp...
- 14 Views
- 0 replies
- 0 kudos
- 20 Views
- 0 replies
- 1 kudos
Liquid Clustering and S3 Performance
Are there any performance concerns when using liquid clustering and AWS S3. I believe all the parquet files go in the same folder (Prefix in AWS S3 Terms) verses folders per partition when using "partition by". And there is this note on S3 performa...
- 20 Views
- 0 replies
- 1 kudos
- 17 Views
- 0 replies
- 1 kudos
Databricks Excel Reader
We’ve recently created a new Excel reader function, and I decided to have a play around with it. I’ve used an open dataset for this tutorial, so you can follow along too. Using file available here - https://www.ons.gov.uk/employmentandlabourmarket/pe...
- 17 Views
- 0 replies
- 1 kudos
- 94 Views
- 13 replies
- 1 kudos
Need Help - System tables that contains all databricks users, service principal details !!
Hi all - I am trying to create a dashabord where I need to list down all users and service principals along with groups and understand their databricks usages. Is there any table available in Databricks that contains user, service principal details. ...
- 94 Views
- 13 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi, I can't find any reference to a user system table in our docs. Instead the recommended approach is to use the API to return users, groups and service principals. You can either run this using the Workspace Client if you only have worspace admin p...
- 1 kudos
- 3026 Views
- 6 replies
- 4 kudos
Resolved! DLT-Asset bundle : Pipelines do not support a setting a run_as user that is different from the owner
Hello !We're using Databricks asset bundles to deploy to several environments using a devops pipeline. The service principal running the CICD pipeline and creating the job (owner) is not the same as the SPN that will be running the jobs (run_as).This...
- 3026 Views
- 6 replies
- 4 kudos
Latest Reply
- 4 kudos
Maybe I'm not catching this or missing something else but I've got the following job in one of my demo workspaces:Creator is my user and the job runs as a service principal account. Those are different identities. I got this by deploying the job with...
- 4 kudos
- 37 Views
- 1 replies
- 0 kudos
Databricks Table Protection Features
This article provides an overview of key Databricks features and best practices that protect Gold tables from accidental deletion. It also covers the implications if both the Gold and Landing layers are deleted without active retention or backup. Cor...
- 37 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Thanks for sharing this. Time Travel is applicable all tables in Databricks NOT restricted to gold.
- 0 kudos
- 279 Views
- 7 replies
- 1 kudos
Azure Content Understanding Equivalent
Hi all,I am exploring Databricks services or components that could be considered equivalent to Azure Document Intelligence and Azure Content Understanding.Our customer works with dozens of Excel and PDF files. These files follow multiple template typ...
- 279 Views
- 7 replies
- 1 kudos
Latest Reply
- 1 kudos
I would work, but you will need to specify and manage the ranges or number of header rows manually. You could potentially read the whole sheet in and then write some code that identifies the range that is of interest and cleans it in parsing. My reco...
- 1 kudos
- 38 Views
- 1 replies
- 0 kudos
Why does my MLflow model training job fail on Databricks with an out‑of‑memory error for large datas
I am trying to train a machine learning model using MLflow on Databricks. When my dataset is very large, the training stops and gives an ‘out-of-memory’ error. Why does this happen and how can I fix it?
- 38 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Suheb This happens because during training the entire dataset or large intermediate objects are being loaded into the driver or executor memory, which can exceed the available memory on the cluster, especially when using large DataFrames, collect...
- 0 kudos
- 3432 Views
- 3 replies
- 0 kudos
pyspark.sql.utils.AnalysisException: Non-time-based windows are not supported on streaming DataFrames/Datasets
pyspark.sql.utils.AnalysisException: Non-time-based windows are not supported on streaming DataFrames/DatasetsGetting this error while writing can any one please tell how we can resolve it
- 3432 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Gaurav_784295 ,In Spark, In case of streaming, please use a time based column in window function. Because, In streaming we cant say "last 10 rows", "limit 10" etc. Because streaming never ends. So when you use window, please dont use columns lik...
- 0 kudos
- 27285 Views
- 11 replies
- 13 kudos
input_file_name() not supported in Unity Catalog
Hey, so our notebooks reading a bunch of json files from storage typically use a input_file_name() when moving from raw to bronze, but after upgrading to Unity Catalog we get an error message:AnalysisException: [UC_COMMAND_NOT_SUPPORTED] input_file_n...
- 27285 Views
- 11 replies
- 13 kudos
Latest Reply
- 13 kudos
The reason why the 'input_file_name' is not supported because this function was available in older versions of Databricks runtime. It got deprecated from Databricks Runtime 13.3 LTS onwards
- 13 kudos
- 59 Views
- 0 replies
- 0 kudos
5 Reasons You Should Be Using LakeFlow Jobs as Your Default Orchestrator
I recently saw a business case in which an external orchestrator accounted for nearly 30% of their total Databricks job costs. That's when it hit me: we're often paying a premium for complexity we don't need. Besides FinOps, I tried to gather all the...
- 59 Views
- 0 replies
- 0 kudos
- 849 Views
- 2 replies
- 0 kudos
Inconsistent PYTHONPATH, Git folders vs DAB
Hello Databricks Community,I'm encountering an issue related to Python paths when working with notebooks in Databricks. I have a following structure in my project:my_notebooks - my_notebook.py /my_package - __init__.py - hello.py databricks.yml...
- 849 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
I have a related question.I'm new to Databricks platform. I struggle with PYTHONPATH issue as the original poster raised. I understand using sys.path.append(...) is one approach for notebook. This is acceptable for ad-hoc interactive session, but thi...
- 0 kudos
- 89 Views
- 0 replies
- 0 kudos
Databricks Asset Bundles Direct Mode
There is a new direct mode in Databricks Asset Bundles: the main difference is that there is no Terraform anymore, and a simple state in JSON. It offers a few significant benefits: - No requirement to download Terraform and terraform-provider-databr...
- 89 Views
- 0 replies
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2208 | |
| 1218 | |
| 1062 | |
| 950 | |
| 888 |
Latest from our Blog
[PARTNER BLOG] Zerobus Ingest on Databricks
Introduction TL;DR ZeroBus Ingest is a serverless, Kafka-free ingestion service in Databricks that allows applications and IoT devices to stream data directly into Delta Lake with low latency and mini...
[PARTNER BLOG] Beyond ADLS Limits: File Arrival Triggers for File Updates
Beyond ADLS Limitations: Making File Arrival Triggers Work for Existing File Updates Using a Flag File MechanismThe Flag File MechanismThe Root Problem: Triggers Only Work on “Create”, Not “Modify” Ev...
AI Newsletter Series (Part 1): Parsing Enterprise PDFs Into AI-Ready Data
Enterprise Account macro trends, strategy doc, account and project updates often end up in PDF format. Meanwhile, usage metrics and account-level signals—such as active users, DBU consumption, and use...
Fine-Grained Access Control in Delta Sharing: Dynamic Views vs Partition Filtering
The Challenge: Sharing Data While Maintaining Privacy Boundaries Imagine you're a global retail company with customer order data spanning multiple regions—North America, Europe, and Asia Pacific. You ...
