topic Re: Can the default cluster Serverless of Databricks install Scala packages in Data Engineering

Can the default cluster Serverless of Databricks install Scala packages

liu — Thu, 04 Sep 2025 08:15:41 GMT

Can the default cluster Serverless of Databricks install Scala packages

I need to use the spark-sftp package, but it seems that serverless is different from purpose compute, and I can only install python packages?
There is another question. I can use purpose compute to put sftp files into dbfs through the paramiko package. Why do I get an error saying the folder cannot be found when using serverless to put sftp files into External Data(s3)

Re: Can the default cluster Serverless of Databricks install Scala packages

-werners- — Fri, 05 Sep 2025 12:28:09 GMT

no scala, you can't even run scala notebooks.

about the sftp: the serverless compute is way more limited than general purpose clusters.
what folder can't be found? dbfs or s3?

Re: Can the default cluster Serverless of Databricks install Scala packages

liu — Thu, 11 Sep 2025 02:41:43 GMT

@-werners-
Thank you for your reply

dbfs and s3 both

Although I found a new method to fulfill my needs, unfortunately, there are still some issues.