OSError: Invalid argument when attempting to save a pandas dataframe to csv

kll — Thu, 30 Mar 2023 17:37:53 GMT

I am attempting to save a pandas DataFrame to as csv to a directory I created in Databricks workspace or in the `cwd`.

import pandas as pd
 
import os
 
df.to_csv("data.csv", index=False)
 
df.to_csv(str(os.getcwd()) + "/data.csv", index=False)

OSError: [Errno 22] Invalid argument
 
---------------------------------------------------------------------------
 
OSError                                   Traceback (most recent call last)
 
File :2
 
      1 # Save CSV to Databricks filesystem
 
----> 2 df.to_csv("pmf_remax.csv", index=False)

I am also saving a file using `_file` ` method and while it doesn't throw an error, i don't see the file in the `cwd`.

***.to_file(str(os.getcwd()) + '/us_df.geojson', driver='GeoJSON')

when i save a file, where does it save it? I tried to run the command `find . -name example.txt` from the root directory in the web terminal and didn't find it.

Re: OSError: Invalid argument when attempting to save a pandas dataframe to csv

Ajay-Pandey — Fri, 31 Mar 2023 11:27:40 GMT

Hi @Keval Shah ,

You can save your dataframe to csv in dbfs storage.

Please refer below code that might help you-

df = pd.read_csv(StringIO(data), sep=',')
#print(df)
df.to_csv('/dbfs/FileStore/ajay/file1.txt')

topic OSError: Invalid argument when attempting to save a pandas dataframe to csv in Data Engineering

OSError: Invalid argument when attempting to save a pandas dataframe to csv

Re: OSError: Invalid argument when attempting to save a pandas dataframe to csv