- 3186 Views
- 5 replies
- 5 kudos
Resolved! Slow imports for concurrent notebooks
Hello all,I have a large number of light notebooks to run so I am taking the concurrent approach launching notebook runs with dbutils.notebook.run in parallel. The more I increase parallelism the more I see the duration of each notebook increasing.I ...
- 3186 Views
- 5 replies
- 5 kudos
Latest Reply
- 5 kudos
Hello @Kaniz Fatma​ yes it is clear.Following some tests on my side using a ***** notebook that all it does is importing stuff and sleeping for 15 secs (so nothing to do with spark) I figured that even with a 32 cores driver, the fatigue point is clo...
- 5 kudos