-werners-
Esteemed Contributor III

@Pantelis Maroudis​ , every notebook will create it's own sparkcontext, and every context means overhead.

The number of cores is not the only metric but also the memory and disks.

Using this approach will also be a heavier burden on the driver.

View solution in original post