- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-30-2023 10:37 AM
I am attempting to save a pandas DataFrame to as csv to a directory I created in Databricks workspace or in the `cwd`.
import pandas as pd
import os
df.to_csv("data.csv", index=False)
df.to_csv(str(os.getcwd()) + "/data.csv", index=False)
OSError: [Errno 22] Invalid argument
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
File :2
1 # Save CSV to Databricks filesystem
----> 2 df.to_csv("pmf_remax.csv", index=False)
I am also saving a file using `_file` ` method and while it doesn't throw an error, i don't see the file in the `cwd`.
***.to_file(str(os.getcwd()) + '/us_df.geojson', driver='GeoJSON')
when i save a file, where does it save it? I tried to run the command `find . -name example.txt` from the root directory in the web terminal and didn't find it.
- Labels:
-
CSV
-
Databricks notebook
-
Pandas dataframe
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-31-2023 04:27 AM
Hi @Keval Shah ,
You can save your dataframe to csv in dbfs storage.
Please refer below code that might help you-
df = pd.read_csv(StringIO(data), sep=',')
#print(df)
df.to_csv('/dbfs/FileStore/ajay/file1.txt')
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-31-2023 04:27 AM
Hi @Keval Shah ,
You can save your dataframe to csv in dbfs storage.
Please refer below code that might help you-
df = pd.read_csv(StringIO(data), sep=',')
#print(df)
df.to_csv('/dbfs/FileStore/ajay/file1.txt')