cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

--files in spark submit task

Kratik
New Contributor III

Regarding --files option in spark submit task of Databricks jobs, would like to understand how it works and what is the syntax to pass multiple files to --files? 
I tried using --files and --py-files and my understanding is, it should make available the mentioned files to driver/executor's execution location. But nothing like that was happening for me. 

There are no proper examples mentioning the syntax as well as working of spark submit job and it is real struggle migrating most of the legacy/on-premise spark jobs which were using spark-submit to Databricks and things are not direct lift and shift and making things complicated. 

1 REPLY 1

Debayan
Esteemed Contributor III

Hi, could you please check if this helps: https://docs.databricks.com/en/files/index.html

Also please tag @Debayan​ with your next response which will notify me, Thank you!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group