- 5368 Views
- 2 replies
- 0 kudos
How to use spark-submit python task with the usage of --archives parameter passing a .tar.gz conda env?
We've been trying to launch a spark-submit python task using the parameter "archives", similar to that one used in Yarn.However, we've not been able to successfully make it work in databricks.We know that for our OnPrem installation we can use som...
- 5368 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
@Ryoji Kuwae Neto :To use the --archives parameter with a conda environment in Databricks, you can follow these steps:1) Create a conda environment for your project and export it as a .tar.gz file:conda create --name myenv conda activate myenv conda...
- 0 kudos