topic Re: Controlling the name of the downloaded csv file from a notebook in Data Engineering

Controlling the name of the downloaded csv file from a notebook

alonisser — Mon, 09 Jun 2025 07:12:25 GMT

I got a notebook with multiple display() commands in various cells, the users are currently downloading the result csv from each cell

I want the downloads to be named after the name of the cell (or any other methods I can make each download have a different name, and not the notebook name which is the same for all)

Bonus points: making the programmatically controllable so I can manipulate the name (add client name from a widget for example)

Re: Controlling the name of the downloaded csv file from a notebook

alonisser — Mon, 09 Jun 2025 09:36:46 GMT

Thanks @james589batson when you write more details "here" is this missing a link? does this solution work with the built in databricks display() or do I need to change the way I turn the df to csv?

Re: Controlling the name of the downloaded csv file from a notebook

Isi — Mon, 09 Jun 2025 11:28:44 GMT

Hey @alonisser

One simple and effective approach is to convert the PySpark DataFrame to a Pandas DataFrame and save it directly to a Unity Catalog Volume with a custom filename.

* You can use widgets to dynamically set values like the client name, and encapsulate the logic in a reusable function to streamline your workflow.

Here’s a minimal example:

import pandas as pd

# Example: replace with dbutils.widgets.get("client_name") if using widgets
client = "001"
section = "transactions"

# Convert Spark DataFrame to Pandas
df_1 = df.toPandas()

# Define a custom filename
filename = f"{client}_{section}.csv"

# Define the target Volume path
volume_path = f"/Volumes/<catalog>/<schema>/<volume_name>/{filename}"

# Export the file as CSV to the Volume
df_1.to_csv(volume_path, index=False)

print(f"File saved to: {volume_path}")

Hope this helps, 🙂

Isi

Re: Controlling the name of the downloaded csv file from a notebook

alonisser — Mon, 09 Jun 2025 20:06:32 GMT

Thanks @Isi this looks like an interesting path, but then, how do we make it easily downloadable?

Re: Controlling the name of the downloaded csv file from a notebook

Isi — Tue, 10 Jun 2025 13:02:06 GMT

Hey @alonisser

Once the file is stored in the volume —whether in S3, GCS, or ADLS— you’ll be able to see it with a custom name defined by the customer or project. Additionally, the files may be saved in different folders, making it easier to identify and re-download the correct one if needed.

This structure allows you to quickly recognize the appropriate file. You can also use Spark or SQL to query the file directly in case you need to review its contents or validate the output without moving or reprocessing the data.

Hope this helps, 🙂

Isi