cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

ETL job execution failure in serverless compute

furqanraza
New Contributor

yesterday

Hi,

We facing an issue when executing the ELT pipeline on serverless compute. In the ETL pipeline, for some users, a task gets stuck every time we run the job. However, a similar ETL pipeline works fine for other users. Furthermore, there are canceled queries in the query history, but it shows that these are still being processed.

Have you faced a similar issue in Databricks with serverless compute?

 

Labels:
0 Kudos
2 REPLIES 2

ozaaditya
New Contributor II

yesterday

Hi,

Could you please share the error message or any logs you’re seeing when the task gets stuck? This will help in diagnosing the issue and identifying potential solutions.

0 Kudos

furqanraza
New Contributor
In response to ozaaditya

10 hours ago

Hi,

The following are the screenshots of query history and query profile.

 

Screenshot 2024-11-29 at 11.06.11.pngScreenshot 2024-12-04 at 09.36.37.png

0 Kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Announcements