cancel
Showing results for 
Search instead for 
Did you mean: 
Community Discussions
Engage in vibrant discussions covering diverse learning topics within the Databricks Community. Explore learning resources, share insights, and collaborate with peers to enhance your skills in data engineering, machine learning, and more.
cancel
Showing results for 
Search instead for 
Did you mean: 

Browse the Community

Certifications

Join dynamic discussions on Databricks certifications within the Community. Exchange insights, tips,...

938 Posts

Training offerings

Explore discussions on Databricks training programs and offerings within the Community. Get insights...

213 Posts

Get Started Discussions

Start your journey with Databricks by joining discussions on getting started guides, tutorials, and ...

2749 Posts

Activity in Community Discussions

mouni2
by > New Contributor II
  • 300 Views
  • 4 replies
  • 3 kudos

Databricks workspace issue

Hello Databricks Support Team,I am unable to access my Databricks workspace due to a login redirect loop.When I sign in, my Databricks account is displayed correctly. However, when I click on the account/workspace, I am redirected back to the Databri...

  • 300 Views
  • 4 replies
  • 3 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 3 kudos

Hi @mouni2, Based on your description, your original account was likely a Databricks Community Edition account. Community Edition was retired on January 1, 2026, and existing Community Edition workspaces are no longer accessible. This is the most com...

  • 3 kudos
3 More Replies
gokkul
by > New Contributor II
  • 305 Views
  • 3 replies
  • 1 kudos

How to access a delta table in UC from lakebase postgres ?

Hi DB Community ,Is there any way to access/write to delta table in UC from lakebase postgres ? There's a way using "Sync Table" - but it is recommended only to read data from Sync Table . Databricks recommends against writing to sync table . Or else...

  • 305 Views
  • 3 replies
  • 1 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 1 kudos

Hi @gokkul, There are a few different patterns here depending on which direction you need data to flow. Let me break them down. READING DELTA/UC DATA FROM LAKEBASE (UC to Postgres) Synced tables are the primary mechanism for this. They automatically ...

  • 1 kudos
2 More Replies
spjti
by > New Contributor
  • 152 Views
  • 2 replies
  • 1 kudos

Manager hierarchy

for employee manager hierarchy some employees are not having correct manager mapping, what can be the reason?

  • 152 Views
  • 2 replies
  • 1 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 1 kudos

Hi @spjti, Incorrect manager mappings in an employee hierarchy typically come from a few common root causes. Since the question is fairly open-ended, here is a checklist of the most likely issues and how to diagnose them in Databricks. COMMON REASONS...

  • 1 kudos
1 More Replies
chris84
by > New Contributor II
  • 195 Views
  • 3 replies
  • 0 kudos

Attempting to load a JSON file fails due to schema issue (Free Edition)

Hello,I created a Volume named 'test_volume' under catalog:workspace and schema:default.Then I uploaded a file named user_0.json into test_volume (fake data, of course):Now I want to load that file into a data frame.With Python in a notebook:With SQL...

chris84_1-1767792178558.png chris84_0-1767792125532.png chris84_2-1767792435608.png
  • 195 Views
  • 3 replies
  • 0 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 0 kudos

Hi @chris84, You already identified the root cause: the JSON file was pretty-printed across multiple lines. By default, Spark's JSON reader expects one JSON record per line (sometimes called "JSON Lines" or NDJSON format). When it encounters a pretty...

  • 0 kudos
2 More Replies
Smriti2
by > New Contributor II
  • 310 Views
  • 6 replies
  • 0 kudos

Same Delta Table, Different Behavior: Dev vs Prod Workspace in Databricks

I recently ran into an interesting Databricks behavior while implementing a row-count comparison using Delta Time Travel (VERSION AS OF).Platform: AzureScenario:Same Unity CatalogSame fully qualified tableSame table ID, location, and Delta formatYet ...

  • 310 Views
  • 6 replies
  • 0 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 0 kudos

Hi @Smriti2, The error "TIME TRAVEL is not allowed. Operation not supported on Streaming Tables" when the table is not intentionally a streaming table typically points to the table having pipeline-related metadata attached to it in the prod workspace...

  • 0 kudos
5 More Replies
Jim_Anderson
by Databricks Employee
  • 32 Views
  • 0 replies
  • 0 kudos

FAQ for Virtual Learning Festival: 16 March - 03 April 2026

General   Q: Where can I ask questions if I have questions while going through the self-paced material? The Databricks Academy Learners Group page is a forum within the Databricks Community where you can ask questions and get them answered by your fe...

Screenshot 2024-11-26 at 13.37.42.png
  • 32 Views
  • 0 replies
  • 0 kudos
saptarshi
by > New Contributor
  • 131 Views
  • 2 replies
  • 0 kudos

Resolved! Certificate Not Received for Databricks Data Engineer Associate Even After 3 Days

Hello,I appeared for the Databricks Data Engineer Associate exam on March 03, 2026.I have not received the certificate yet, even though I have received a confirmation of passing the exam on email with section wise scores.I tried retrieving the certif...

  • 131 Views
  • 2 replies
  • 0 kudos
Latest Reply
saptarshi
New Contributor
  • 0 kudos

Thanks, I received the certificate just now

  • 0 kudos
1 More Replies
Schusmeicer
by > New Contributor II
  • 137 Views
  • 3 replies
  • 0 kudos

Unity Catalog Error: PERMISSION_DENIED: Can not move tables across arclight catalogs (Free Edition)

Subject: Unity Catalog Error: PERMISSION_DENIED: Can not move tables across arclight catalogs (Free Edition)Body: Hi everyone,I'm trying to set up a Spark Declarative Pipeline (SDP) using a streaming table on Databricks Free Edition, but I'm hitting ...

  • 137 Views
  • 3 replies
  • 0 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 0 kudos

Hi @Schusmeicer, The "Can not move tables across arclight catalogs" error you are seeing is specific to how Unity Catalog is managed on the Databricks Free Edition. "Arclight" is the internal infrastructure name for the Free Edition's managed Unity C...

  • 0 kudos
2 More Replies
crami
by > New Contributor III
  • 277 Views
  • 4 replies
  • 0 kudos

Databricks Dataframe: Merge exception for string column in delta table with change detectio enabled.

Hi,I am trying to append change data records into delta table using dataframe. However, I am encountering an issue while appending new records to existing data in delta table. The exception raised isCannot merge columns 'ColumnX' into 'ColumnX'Both n...

  • 277 Views
  • 4 replies
  • 0 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 0 kudos

Hi @crami, The "Cannot merge columns 'ColumnX' into 'ColumnX'" error you are seeing is caused by passing "delta.enableChangeDataFeed" as a write option on the DataFrame writer. Change Data Feed (CDF) is a table-level property, not a per-write option....

  • 0 kudos
3 More Replies
fehrin1
by > New Contributor III
  • 243 Views
  • 13 replies
  • 3 kudos

nookup in web console

In the DB free web console on nslookup de.wikipedia.org i getREFUSEDIs this a hard restriction of the free edition or can i work around this?In some documentation i found that the free edition has external access to a limited set of sites only, is th...

  • 243 Views
  • 13 replies
  • 3 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 3 kudos

Hi @fehrin1, This is expected behavior on the Databricks Free Edition. The Free Edition documentation confirms that "outbound internet access is restricted to a limited set of trusted domains." https://docs.databricks.com/aws/en/getting-started/free-...

  • 3 kudos
12 More Replies
ruicarvalho_de
by > New Contributor III
  • 84 Views
  • 1 replies
  • 0 kudos

Inconsistent UNRESOLVED_COLUMN._metadata error on Serverless compute during MERGE operations

Hi.I've been facing this problem for the last couple of days.We're experiencing intermittent failures with the error [UNRESOLVED_COLUMN.WITHOUT_SUGGESTION] A column, variable, or function parameter with name '_metadata' cannot be resolved. SQLSTATE: ...

  • 84 Views
  • 1 replies
  • 0 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 0 kudos

Hi @ruicarvalho_de, The behavior you are seeing is related to how the _metadata virtual column interacts with Spark Connect (which Serverless compute uses under the hood) and how deferred analysis changes when that hidden column is resolved. Here is ...

  • 0 kudos
aranjan99
by > Contributor
  • 193 Views
  • 5 replies
  • 0 kudos

databricks list job runs api limit doesnt work

noticed that the limit parameter on the list job runs api: https://docs.databricks.com/api/workspace/jobs/listrunsdoesnt work. no matter what limit i set, i always get all the runs which can be too many if i want all runs.If i set offset as > 1, then...

  • 193 Views
  • 5 replies
  • 0 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 0 kudos

Hi @aranjan99, To add some detail to the earlier reply: the confusion here comes from the fact that the REST API and the Databricks CLI handle the "limit" parameter differently. REST API BEHAVIOR In the REST API (POST or GET to /api/2.1/jobs/runs/lis...

  • 0 kudos
4 More Replies
ShyamalaT
by > New Contributor
  • 82 Views
  • 3 replies
  • 1 kudos

Access Lab

Hi Team,How do I access lab? Do I need to take subscription?

  • 82 Views
  • 3 replies
  • 1 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 1 kudos

Hi @ShyamalaT, As @Advika and @szymon_dybczak mentioned, the hands-on labs that accompany Databricks Academy courses do require either an Instructor-Led Training (ILT) enrollment or a Databricks Academy Labs subscription. Here is a quick summary of y...

  • 1 kudos
2 More Replies
CuriousEngineer
by > New Contributor
  • 163 Views
  • 4 replies
  • 0 kudos

Row count is higher than expected after breaking cases into 15-minute segments

I’m building a fact table in Databricks where the grain is one row per case per 15-minute segment. For example, if a case lasts 90 minutes, I expect about 6 rows.I generate the 15-minute segments and MERGE into a Delta table (incremental load). Howev...

Get Started Discussions
Databricks
Databricks SQL
pyspark
  • 163 Views
  • 4 replies
  • 0 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 0 kudos

Hi @CuriousEngineer, This is a common scenario, and there are good solutions. Nice job including the code and the minimal reproducible example -- that makes it much easier to help. There are two places in your query that are very likely introducing d...

  • 0 kudos
3 More Replies
Kirankumarbs
by > New Contributor III
  • 214 Views
  • 8 replies
  • 2 kudos

Continuous Job - How to set max_retries

Hello Community,I have a couple of continuous workflows (jobs) running in production, and they’ve been working well so far. However, we’re seeing some transient failures that are causing the entire job to restart — which I’d prefer to avoid.While we ...

Screenshot 2026-02-17 at 07.52.23.png
  • 214 Views
  • 8 replies
  • 2 kudos
Latest Reply
SteveOstrowski
Databricks Employee
  • 2 kudos

Hi @Kirankumarbs, Thanks for posting this. I can see from the thread that the existing replies confirmed the limitation but did not offer much in the way of alternatives. Let me provide a fuller picture. THE SHORT ANSWER You cannot set a custom max_r...

  • 2 kudos
7 More Replies