We are running a Free Edition Hackathon from November 5-14, 2025 and would love for you to participate and/or help promote it to your networks. Leverage Free Edition for a project and record a five-minute demo showcasing your work. Free Edition launc...
- 793 Views
- 1 replies
- 7 kudos
2 weeks ago
As data volume and complexity increase, engineers are left figuring out how to manage, monitor and maintain fragile pipelines while also handling fragmented tools. The Big Book of Data Engineering equips you with cutting-edge methods for building pip...
- 309 Views
- 0 replies
- 4 kudos
Wednesday
How to Register & Prepare If you're interested in advancing your skills with Databricks through a Specialist Session, here's a clear guide on how to register and what free courses you can take to prepare effectively. How to Begin Your Learning Path S...
- 2043 Views
- 0 replies
- 7 kudos
10-02-2025
This week was charged – from insightful discussions to Hackathon brilliance, our members showed off their creativity, collaboration, and unmatched Databricks energy! Let’s dive into this week’s highlights Community Stars of the Week Big cheers to e...
- 186 Views
- 2 replies
- 4 kudos
Friday
Hello Community, Let me start off with a quick question: Have you ever... Migrated your workloads from on-prem Spark to Databricks and encountered a bug and thought, “I wish I could repro this locally to debug the issue without burning cluster hours...
- 335 Views
- 4 replies
- 8 kudos
Thursday
Community Activity
- 34 Views
- 4 replies
- 0 kudos
Accessing permanent file system from Databricks App
I have a Python-based FastAPI application with a React frontend, which I would like to deploy to Databricks apps. I'm able to do it, but having trouble with file system access. In addition to database access, my application relies on some file operat...
- 34 Views
- 4 replies
- 0 kudos
Latest Reply
- 0 kudos
Related thread, but would like to confirm:https://community.databricks.com/t5/administration-architecture/how-to-access-unitycatalog-s-volume-inside-databricks-app/td-p/121716
- 0 kudos
- 3082 Views
- 3 replies
- 0 kudos
What is the best way to connect Oracle CRM cloud from databricks?
What is the best way to connect Oracle CRM cloud from databricks?
- 3082 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Oracle CRM on Demand is a Cloud platform not a relational database.Is there any proven solution for this requirement?
- 0 kudos
- 110 Views
- 2 replies
- 2 kudos
Dynamic populated multiple choice query parameters - Documentation clarification
Dear colleagues,I'm in need of finishing a dashboard with the new databricks dashboard structure, released somewhat recently. On it, I'll have some calculations which need to be percentages over certain partitions, such as `input_amount_percentage_pe...
- 110 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
It is possible to create dynamic parameter lists in Databricks dashboards—so, yes, the documentation is correct, but the crucial detail is that not all data sources or query environments support this feature in the exact same way, and certain limitat...
- 2 kudos
- 24629 Views
- 9 replies
- 6 kudos
Resolved! Run workflow using git integration with service principal
Hi,I want to run a dbt workflow task and would like to use the git integration for that. Using my personal user I am able to do so but I am running my workflows using a service principal.I added git credentials and the repository using terraform. I a...
- 24629 Views
- 9 replies
- 6 kudos
Latest Reply
- 6 kudos
Databricks has updated documentation on authorizing a service principal to access Git folders.Now Databricks has 3 different options to run the jobs by pointing to the Git code.1. User PAT - Configure Git credentials & connect a remote repo to Databr...
- 6 kudos
- 11 Views
- 0 replies
- 0 kudos
Enable Compute Policy Management and Compute Policy Admin Role
Hi,I have an account with an Enterprise plan and wanted to change some features of the compute policy for a cluster i wanted to create in a workspace I am an Admin of. But I cannot because the fields are read-only.Co-Pilot directed me to look for an ...
- 11 Views
- 0 replies
- 0 kudos
- 15 Views
- 0 replies
- 0 kudos
Databricks Model Serving Endpoint Fails: “_USER not found for feature table”
Hi Databricks Community,I’m trying to deploy a model serving endpoint that uses Databricks Feature Store (Unity Catalog, online tables).My offline and online feature tables are created and visible in Databricks.The model is logged with FeatureEnginee...
- 15 Views
- 0 replies
- 0 kudos
- 35 Views
- 5 replies
- 3 kudos
Resolved! Accessing Databricks data in Salesforce via zero copy
I have uploaded clickstream data as shown belowDo i have to mandatorily share via Delta sharing for values to be exposed in Salesforce ?At the Salesforce end i have confirmed that i have a working connector where i am able to see samples data , but u...
- 35 Views
- 5 replies
- 3 kudos
Latest Reply
- 3 kudos
So, for instance I have catalog called project in databricks free edition. If I would like to assign proper permission for my Service Principal (so that it can see the tables wihtin catalog and can query them) first I need to set 2 preequisite permis...
- 3 kudos
- 9540 Views
- 5 replies
- 6 kudos
Resolved! Limit on number of result rows displayed on databricks SQL UI
Databricks SQL UI currently limits the query results display to 64000 rows. When will this limit go away? Using SSMS I get 40MM rows results in the UI and my users won't switch to databricks SQL for this reason
- 9540 Views
- 5 replies
- 6 kudos
Latest Reply
- 6 kudos
want to understand the reason behind it. retrieving 500k records is not a problem, is it rendering over the http thats the problematic?
- 6 kudos
- 23535 Views
- 4 replies
- 23 kudos
Getting Started with Databricks - Build a simple Lakehouse analytics pipeline
Getting started with Databricks - Build a simple Lakehouse analytics pipeline The demo example in this guide illustrates a lakehouse analytics pipeline using the well-known NYC taxi trip dataset. This public dataset is also available at Kaggle. This...
- 23535 Views
- 4 replies
- 23 kudos
- 52 Views
- 2 replies
- 0 kudos
Azure databrics Learning tutorials ADB+SQL,ADB+PYSPARK,ADB+PYTHON
Suggest me the best learning tutorials of Azure databricks with the combinations of pyspark,python,sql is the any learning web based tutorials from Databricks suggest me the best one from scratch to advanced
- 52 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @vijaypodili ,I can recommend Data Engineering Learning Path on Databricks Academy:https://customer-academy.databricks.com/On Udemy, there's an excellent course that covers all the important aspects of working with Databricks on a daily basis:Data...
- 0 kudos
- 39 Views
- 3 replies
- 2 kudos
Can anyone share Databricks security model documentation or best practice references
Can anyone share Databricks security model documentation or best-practice references
- 39 Views
- 3 replies
- 2 kudos
Latest Reply
- 2 kudos
Very broad Qs and you have already received reply for the various reference materials. Please share the specific scenario if you are looking for a pointed guideance from the community
- 2 kudos
- 17 Views
- 4 replies
- 2 kudos
Lakeflow jobs
Hi I am currently working on migrating all ADF jobs to LakeFlow jobs. I have a few questions:Pipeline cost: What is the cost model for running LakeFlow pipelines? Any documentation available? ADF vs Lakeflow Job?Job reuse: Do LakeFlow jobs reuse the...
- 17 Views
- 4 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi @Nidhig ,1. Regarding pipeline cost - here you're mostly paying for compute usage. So the exact price depends on which plan you are at and which cloud provider you are using. For instance, for Azure premium plan and US East region you have followi...
- 2 kudos
- 33 Views
- 2 replies
- 1 kudos
How to make streaming files?
Thanks for reviewing my threads.I am trying to test streaming table /files in databricks FREE edition.-- Create test streaming tableCREATE OR REFRESH STREAMING TABLE user.demo.test_bronze_st ASSELECT * FROM STREAM read_files('/Volumes/xxx_ws/demo/raw...
- 33 Views
- 2 replies
- 1 kudos
Latest Reply
- 1 kudos
Thanks for weighing in. Are you saying CREATE OR REFRESH STREAMING TABLE user.demo.test_bronze_st cannot be used in FREE Edition?If we can use it, how do to create STREAM read_files('/Volumes/xxx_ws/demo/raw_files/test.csv'),where .csv sitting on lo...
- 1 kudos
- 25 Views
- 3 replies
- 0 kudos
Custom Catalog Not Visible to ADF and Looker in Databricks
Hi Databricks community,I’ve created a catalog called azure_data_factory and a database called studytube_azure_mysql_production. Within that database, I have three tables: academies, academy_naming_settings, and users.See the left side of the screens...
- 25 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
As mentioned by @szymon_dybczak, it is certainly a 'Grant' issue. Pls try granting the USE permissions to your SP first. Hope that resolves your issue.
- 0 kudos
- 21 Views
- 1 replies
- 0 kudos
Can’t save results to target table – out-of-memory error
Hi team, I’m processing ~5,000 EMR notes with a Databricks notebook. The job reads from `crc_lakehouse.bronze.emr_notes`, runs SciSpaCy UMLS entity extraction plus a fine-tuned BERT sentiment model per partition, and builds a DataFrame (`df_entities`...
- 21 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
You’re right that the behaviour is weird at first glance (“5k rows on a 64 GB cluster and I blow up on write”), but your stack trace is actually very revealing: this isn’t a classic Delta write / shuffle OOM – it’s SciSpaCy/UMLS falling over when loa...
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2123 | |
| 1183 | |
| 990 | |
| 948 | |
| 852 |
Latest from our Blog
Upgrade Your HMS and AWS Glue Tables: How-To Guide for Migrating Foreign Tables to Unity Catalog
Intro Unlock Unity Catalog governance and performance by upgrading Hive Metastore (HMS) and AWS Glue foreign tables to Managed Tables using the new Upgrade Foreign Table workflow. Managed Tables provi...
From Lag to Agility: Reinventing Freshworks’ Data Ingestion Architecture
As a global software-as-a-service (SaaS) company specializing in providing intuitive, AI-powered business solutions designed to enhance customer and employee experiences, Freshworks depends on real-ti...
From Hackathon to Impact: Building a Data-Driven Forecasting App on Databricks
IntroductionAfter Databricks' Data + AI Summit, an electrifying hackathon lit up the London HQ. Teams from across industries came together to build bold, tangible solutions using DBX’s latest features...
How to Deploy Metric Views with DABs
Introduction Unity Catalog Metric Views provide the ability to build a semantic layer in Databricks, using YAML and SQL. This shifts the semantic layer left into your catalog, moving it closer to your...
