Hi @carlosna , Unfortunately, if the file was saved to the /databricks/driver
location, it will be lost when the cluster is shut down. The driver node of a Databricks cluster is ephemeral, meaning that its storage is attached to the machine hosting the driver node, and is destroyed when the node is terminated.
To prevent this from happening, you should save your results to a more reliable location, such as DBFS (Databricks file system) or a cloud storage service such as Azure Blob Storage or AWS S3.
You can use DBFS to save data in an S3- like interface. The DBFS provides an abstraction over underlying cloud storage, allowing you to store different types of data such as notebooks, libraries, and data files. The files are persisted independently of the cluster's lifecycle, so you can save files in DBFS even after the cluster has terminated.