Starting from yesterday 17/5/2022 i start getting errors while running notebooks or jobs on clusters of Databricks GCP. The error is: SparkException: Environment directory not found at /local_disk0/.ephemeral_nfs/cluster_libraries/pythonThe job/noteb...
Thank you for the answer - i will start checking the changes.I couldn't find the logFileCommitBufferSize parameter in the databricks configuration. Can you give me a link? What should be the value as example:spark.databricks.delta.logFileCommitBuffer...
Databricks supports detected an issue with the NFS mounts on GCP. Looks like DBR 10.X versions were affected. After several hours they fixed it and now the same clusters are back to normal.
