cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 
Community Bricktalk | Master Multi-Cloud Data Sharing with OpenSharing & Global Distribution

Hey Databricks Community! Join us for a free community-sponsored BrickTalk where we’ll explore Multi-Cloud Data Sharing with OpenSharing SecureConnect & Global Distribution. BrickTalks is a community event series where Databricks experts share real-w...

  • 346 Views
  • 0 replies
  • 0 kudos
Wednesday
Databricks Advanced Learning Festival: November 10 - December 08 2026

Databricks Advanced Learning Festival November 10 – December 8, 2026 Join us for a four-week event dedicated to learning, upskilling, and advancing your career in data engineering, analytics, machine learning, and generative AI. Whether you are ...

  • 41905 Views
  • 72 replies
  • 13 kudos
10-08-2026
✈️ 9 Cities, 1 Epic November: The Data + AI World Tour is Here!

NOVEMBER · THE TOUR IS ON THE MOVE Nine stops. Three regions.One epic November. THIS MONTH'S ROUTE Munich → Atlanta → Amsterdam → London → Toronto → Los Angeles (Anaheim) → New York City → Milan → Tokyo A free, one-day event on how organizations...

  • 3338 Views
  • 1 replies
  • 1 kudos
3 weeks ago
DATA + AI LIVE | Context Is Just the Beginning: Meet your data smart AI coworker

DATA + AI LIVE Time: AMER Nov 11 / 9 AM PT / 12 PM ET APAC Nov 12 / 9:30 AM IST / 12 PM SGT / 1 PM JST-KST / 3 PM AEDT EMEA Nov 17 / 9 AM GMT / 10 AM CET Register for the Workshop Note: Marking RSVP on this Community event does not register you fo...

  • 2543 Views
  • 0 replies
  • 2 kudos
4 weeks ago
DATA + AI World Tour 2026

REGISTRATION IS NOW OPEN The Data + AI World Tour is heading your way A free, one-day event on how organizations are building AI they can trust to deliver real outcomes - by grounding models in their own enterprise data and business context. THE...

  • 4710 Views
  • 0 replies
  • 4 kudos
09-01-2026
Solution Accelerator Series | Predict Patient Disease Risk & Measure Quality

Predicting patient disease risk and measuring quality of care are important to precision health. This Solution Accelerator demonstrates a machine learning workflow using healthcare data to identify clinical and demographic factors associated with dis...

  • 185 Views
  • 0 replies
  • 0 kudos
Wednesday
🌟 Community Pulse: Your Community Roundup! September 14 – October 11, 2026

Recap • September 14 – October 11 Community Pulse Four weeks of articles, learning content, and conversations – the highlights, all in one read. Community Stars   The members who showed up and made a real difference : @ivanvyd @data_pulse @Tho...

  • 1097 Views
  • 0 replies
  • 4 kudos
10-15-2026

Community Activity

MichaelAnckaert
by > • New Contributor
  • 165 Views
  • 3 replies
  • 0 kudos

Issue with CDC ingestion pipeline

Hello everyoneI'm trying to setup an ingestion pipeline from a mysql server and keep running into an issue. The setup was validated and the pipeline created. When I run the pipeline the destination tables are created in the schema, but then the follo...

  • 165 Views
  • 3 replies
  • 0 kudos
Latest Reply
ShamenParis
Contributor III
  • 0 kudos

Hi @MichaelAnckaert ,The error Input URI is not a valid UC Path: s3:/ indicates that Unity Catalog is trying to physically create the destination tables (or the pipeline staging area) in AWS S3, but the storage path it has been handed is malformed. A...

  • 0 kudos
2 More Replies
Sunioj
by > • New Contributor
  • 128 Views
  • 2 replies
  • 0 kudos

Issues making requests to Databricks from AWS Lambda

Hello,I am getting acquainted with Databricks, of which I have signed up for a free account. I have a Lambda function using Python 3.14 and the databricks-sql-connector library that runs on a schedule, to take data from a file in S3 and put the diffe...

  • 128 Views
  • 2 replies
  • 0 kudos
Latest Reply
ShamenParis
Contributor III
  • 0 kudos

Hi @Sunioj ,Both of your errors are caused by Databricks Free Edition compute limits and how databricks-sql-connector handles warehouse startups:Error 1 (BAD_REQUEST: Cannot create the resource): Free Edition accounts only get one 2X-Small Serverless...

  • 0 kudos
1 More Replies
suranga
by > • Databricks Partner
  • 37 Views
  • 0 replies
  • 0 kudos

Why your materialized view keeps doing full recomputes, and how to find out

This came out of a community thread about serving gold materialized views to Power BI. Everything downstream kept getting rewritten, and the root cause turned out to be simple: most of the MVs were being fully recomputed on every update. A full recom...

  • 37 Views
  • 0 replies
  • 0 kudos
Daniel_B
by > • Contributor II
  • 43 Views
  • 0 replies
  • 0 kudos

DevOps Fundamentals: Demo: Deploying the Databricks Project

The course mentions that course material may lag behind the current version of DataBricks.I wonder if the difference between the video in Demo: Deploying the Databricks Project is now too different to what is seen in the Vocareum videos.For example: ...

  • 43 Views
  • 0 replies
  • 0 kudos
suranga
by > • Databricks Partner
  • 36 Views
  • 0 replies
  • 0 kudos

Federated PostgreSQL timestamps in Unity Catalog: a view that gives everyone the same answer

This came out of a community thread about a foreign catalog over PostgreSQL where daily counts in Databricks didn't match Postgres, and setting the session time zone made things more confusing, not less. I got part of it wrong the first time, so here...

  • 36 Views
  • 0 replies
  • 0 kudos
AdnanFaridKhan
by > • New Contributor II
  • 320 Views
  • 4 replies
  • 1 kudos

Masking Columns for a Client Workspace When You Can’t Use Catalog Binding

Hi Everyone,I am Adnan and I am a Cloudera Platform expert recently joined the Databricks Community. I will be sharing all my implementations and experience I have gained as Big Data Expert and any solutions I implement to address my current business...

  • 320 Views
  • 4 replies
  • 1 kudos
Latest Reply
suranga
Databricks Partner
  • 1 kudos

Hi @rachelwilson,I'd do both, because they catch different failures.In the sync job, as the last step. It runs right after every sync, so a tag that failed to copy is caught in the same run, before anyone queries the table.A separate sweep on its own...

  • 1 kudos
3 More Replies
rajkumar8k
by > • New Contributor II
  • 56 Views
  • 0 replies
  • 0 kudos

Databricks Latest Features: Genie Code CLI, App Builder, Agent Memory and DABs Deployment History

Hi everyone,Databricks has shipped a lot of new features recently. Here’s a quick summary of the most useful ones, grouped by area.1. DABs Deployment HistoryWhat it is:Databricks Asset Bundles (DABs) now keep a history of every deployment. Each recor...

  • 56 Views
  • 0 replies
  • 0 kudos
AlexMorgan
by > • New Contributor
  • 253 Views
  • 4 replies
  • 1 kudos

How do you test data quality in a Delta Lake pipeline?

I'm exploring practical approaches to validating data quality in Databricks pipelines.For pipelines that ingest data into Delta tables, what checks do you typically automate at each stage?I'm particularly interested in handling:Schema changes and sch...

  • 253 Views
  • 4 replies
  • 1 kudos
Latest Reply
rajkumar8k
New Contributor II
  • 1 kudos

In my experience a combination works best:DLT / Lakeflow expectations check data quality while the pipeline runs. Use warn, drop or fail depending on how serious the issue is, and send bad records to a quarantine table instead of losing them.PySpark ...

  • 1 kudos
3 More Replies
BabuMahesh
by > • New Contributor
  • 1395 Views
  • 1 replies
  • 0 kudos

Databricks & Bigquery

Databricks is packaging a old version of big-query jar(Databricks also repackaged and created a fat jar), and our application needs a latest jar. Now the latest jar depends on spark-bigquery-connector.properties  file for a property scala.binary.vers...

  • 1395 Views
  • 1 replies
  • 0 kudos
Latest Reply
oliverstonez
New Contributor III
  • 0 kudos

Classloader shadowing in Databricks Runtime (DBR) is a common headache when the built-in connector JARs take precedence over cluster-installed libraries.Instead of manually stripping and rebuilding the JAR, try setting spark.driver.userClassPathFirst...

  • 0 kudos
TheDataMaverick
by > • Databricks Partner
  • 51 Views
  • 0 replies
  • 0 kudos

The Databricks Architect in the Age of AI: Why Judgement Matters More Than Knowing Every Feature

The Databricks Architect in the Age of AI: Why Judgement Matters More Than Knowing Every FeatureAs AI makes technical knowledge increasingly accessible, the real differentiator in enterprise data architecture is shifting towards experience, context a...

  • 51 Views
  • 0 replies
  • 0 kudos
jooho_yeo
by • Databricks Employee
  • 1410 Views
  • 1 replies
  • 1 kudos

Introducing 5XL SQL Warehouses: A Practical Guide to Meeting SLAs for Your Most Demanding Workloads

While data volumes are growing, the time windows to process them are getting stricter. Enterprise customers are generating more data than ever, and the speed is only accelerating with more agentic workflows. At the same time, the SLAs for when it nee...

Screenshot 2026-05-07 at 5.25.56 AM.png
  • 1410 Views
  • 1 replies
  • 1 kudos
Latest Reply
suryaprayaga
Contributor II
  • 1 kudos

Thanks for sharing the article. My question:How do you enable this on Community edition of the Databricks?  Thanks

  • 1 kudos
DB1To3
by > • Valued Contributor II
  • 414 Views
  • 7 replies
  • 4 kudos

Resolved! Missing spark structured streaming metadata blobs?

I'm exploring spark structured streaming, and contemplating uses for it.I'd like to launch a job on a recurring basis from an external scheduler.  It will launch a spark job with trigger (availableNow=True). I'm concerned about overlapping executions...

DB1To3_0-1794341533309.png
  • 414 Views
  • 7 replies
  • 4 kudos
Latest Reply
saurabh18cs
Honored Contributor III
  • 4 kudos

Hi @DB1To3 Just make this concurrent run to 1 for your streaming jobs are your problem is sorted , it wont corrupt checkpoint locatiion  

  • 4 kudos
6 More Replies
2nonto
by > • New Contributor
  • 137 Views
  • 1 replies
  • 0 kudos

Pushing a notebook with outputs to Github

I am experiencing an error when trying to commit output and Databricks notebook to GitHub. The error message states, "Error creating or updating commit outputs config file."Could you please assist me in resolving this issue?

Screenshot 2026-11-11 214121.png
  • 137 Views
  • 1 replies
  • 0 kudos
Latest Reply
Yougasundar
Contributor
  • 0 kudos

Hi @2nonto,That message comes from the step that creates the .databricks/commit_outputs file, which controls whether the outputs of .ipynb notebooks are committed. Please check these points in order. 1) Committing outputs is off by default. A workspa...

  • 0 kudos
itsme_gk
by > • New Contributor
  • 120 Views
  • 1 replies
  • 0 kudos

Free edition - serverless option stopped working

I am on free edition and suddently serverless option stopped working.  Is there any restriction on free edition or something is down today?

  • 120 Views
  • 1 replies
  • 0 kudos
Latest Reply
arthurfr23
New Contributor III
  • 0 kudos

The most likely cause is that you hit the Free Edition daily compute quota. It isn't usually an outage. The Free Edition limitations page says that if you go over your quota, your workspace's compute is shut down and stays unavailable for the rest of...

  • 0 kudos
thekushalvishwa
by > • New Contributor
  • 128 Views
  • 1 replies
  • 0 kudos

Is Databricks Losing Its Charm in the Age of AI?

I've been working with Databricks for nearly eight years, and lately I've been reflecting on how our role as data engineers is evolving.One thing I've noticed is that development is increasingly becoming about prompting AI, troubleshooting issues, fi...

  • 128 Views
  • 1 replies
  • 0 kudos
Latest Reply
data_pulse
Contributor
  • 0 kudos

@thekushalvishwa AI can write the pipeline in few seconds. It might not explain the DBU bill to finance. A timely post, and one I suspect many of us relate to.My take is that AI has made writing the code cheap, so our value has moved to the parts it ...

  • 0 kudos
Welcome to the Databricks Community!

Once you are logged in, you will be ready to post content, ask questions, participate in discussions, earn badges, and more.

Spend a few minutes exploring Get Started Resources, Learning Paths, Certifications, and Platform Discussions.

Join Learning Events here in the Community.

Connect with peers through Databricks User Groups and learn more about Community Events happening near you. We’re excited to see you get involved.

Top Kudoed Authors

Latest from our Blog

How to Query Unity Catalog Views from External Engines

Summary External engines can now query Unity Catalog views. Views over Delta tables are served through the Unity REST API (Generally Available). Views over Iceberg tables are served through the Iceber...

569Views 0kudos