cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Serverless Custom Environment Imaging

AlexM
Visitor

Hi,

I'm looking at moving from job clusters to serverless environments. Ideally to reduce cost and improve start up time.
I can see that it is now possible to specify a custom environment .yaml file - and specify Python packages to be installed.

Is there any mechanism to 'pre-install' or use a container image for the serverless? As these packages can take a while to install. I read that there is some caching going on - but the details are a bit opaque?

Would appreciate any advice,

Thanks,
Alex

 

0 REPLIES 0