Hi TimB,

How are you doing today?, As per my understanding, yeah, you got it—serverless jobs don’t keep installed packages, so every time a process runs, it installs the dependencies again, which slows things down. A better way is to package your dependencies into a wheel (.whl) file and install it from DBFS or S3, so you’re not downloading them every time. If your libraries are available in Unity Catalog, you can attach them there so jobs can use them without reinstalling. You could also reduce the number of parallel processes to limit repeated installs, or if possible, switch to a shared job cluster, where dependencies only install once instead of every run. This should help things run much faster—let me know if you need help setting it up! 

Regards,

Brahma