Databricks Community

I faced some challenges in my projects related to parallel processing in Databricks. In many cases, the issue was not the volume of data itself, but the overall execution time. I was processing a relatively small number of objects, but each object re...

Maybe someone has encountered this problem before?I’m running parallel loading for 10 objects using pool.map. Nine of them complete successfully, but one fails when trying to read a configuration file. The problem occurs occasionally and doesn’t foll...