Databricks Community

Within a script, I noticed that the network connection between driver and the mounted network drives is often a huge bottleneck. It seems that the network through speed is unreasonable low for being an Azure Single node: Standard_DS12_v2 · DBR: 14.3....

I receive this respone when trying to download a csv file: adb-737684297741988.8.azuredatabricks.net/ajax-api/2.0/workspace/export?path=/Repos/jhu_admin%40siemensgamesa.com/sgre-dfp-lifestrategy-python/cpo/output/Rattlesnake Flats/Rdata/AssetCheck_Ra...

Whenever I run my script it resets the notebook state:"The spark driver has stopped unexpectedly and is restarting. Your notebook will be automatically reattached.at com.databricks.spark.chauffeur.Chauffeur.onDriverStateChange(Chauffeur.scala:1467)"T...

I have the following code:spark.sparkContext.setCheckpointDir("dbfs:/mnt/lifestrategy-blob/checkpoints") result_df.repartitionByRange(200, "IdStation") result_df_checked = result_df.checkpoint(eager=True) unique_stations = result_df.select("IdStation...