cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Jobs indefinitely pending with libraries install

martindlarsson
New Contributor III

I think I found a bug where you get Pending indefinitely on jobs that has a library requirement and the user of the job does not have Manage permission on the cluster.

In my case I was trying to start a dbt job with dbt-databricks=1.8.5 as library. The cluster did not have the library installed and I had only "Can restart" access to the cluster. When I started the job it was Pending in about 20 minutes until I canceled it and the result in the Runs view Status: Internal error and Error Code: LibraryInstallationError.

Once I gave myself the "Can Manage" on the cluster the job started and the Library successfully installed.

2 REPLIES 2

VZLA
Databricks Employee
Databricks Employee

Thanks for your feedback!

Just checking is this still an issue for you? would you share more details? if I wanted to reproduce this for example.

martindlarsson
New Contributor III

This is not an issue, just wanted to let you know and and if other users has jobs pending forever. I think you have all the information in my description to reproduce.

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now