- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-17-2025 07:08 AM
Hi @cpatte7372 ,
Serverless compute has some limiations. For instnace, you can't install cluster scoped libraries etc.Take a look at full list here:
- Compute policies
- Compute-scoped init scripts
- Compute-scoped libraries, including custom data sources and Spark extensions. Use notebook-scoped libraries instead.
- Instance pools
- Compute event logs
- Most Apache Spark compute configurations. For a list of supported configurations, see Configure Spark properties for serverless notebooks and jobs.
- Environment variables. Instead, Databricks recommends using widgets to create job and task paramers
Serverless compute limitations - Azure Databricks | Microsoft Learn
But you can install notebook-scoped libraries, like I did below: