- 8421 Views
- 2 replies
- 0 kudos
Delta creates more small files during merge and updates operations.
- 8421 Views
- 2 replies
- 0 kudos
Latest Reply
Delta solves the large number of small file problems using the below operations available for a Delta table. Optimize writes helps to optimizes the write operation by adding an additional shuffle step and reducing the number of output files. By defau...
1 More Replies
- 667 Views
- 0 replies
- 0 kudos
Tourism attractions in kerala are described here. Kerala has a rich tourism background, which contributes much to the economy. Tourism is the way of income for a large number of people in Kerala. National parks, wild life sanctuaries, etc. are the ma...
- 667 Views
- 0 replies
- 0 kudos
- 5207 Views
- 4 replies
- 5 kudos
Hello all,I have a large number of light notebooks to run so I am taking the concurrent approach launching notebook runs with dbutils.notebook.run in parallel. The more I increase parallelism the more I see the duration of each notebook increasing.I ...
- 5207 Views
- 4 replies
- 5 kudos
Latest Reply
Hello @Kaniz Fatma yes it is clear.Following some tests on my side using a ***** notebook that all it does is importing stuff and sleeping for 15 secs (so nothing to do with spark) I figured that even with a 32 cores driver, the fatigue point is clo...
3 More Replies
