10-26-2021 01:40 AM
05-24-2022 11:52 PM
Hi
you can try:
my_df = spark.read.format("csv")
.option("inferSchema","true") # to get the types from your data
.option("sep",",") # if your file is using "," as separator
.option("header","true") # if your file have the header in the first row
.load("/FileStore/tables/CREDIT_1.CSV")
display(my_df)
from above you can see that my_df is a spark dataframe and from there you can start with you code.
10-26-2021 01:55 AM
This means the path to the file you typed is not valid, the file is not there.
Can you check in the Data/DBFS page if the file is there (or via dbutils.fs.ls)?
10-26-2021 02:20 AM
10-26-2021 06:51 AM
10-27-2021 06:51 AM
Hi @Venky D , Were you able to read your CSV file using databricks?
08-18-2022 09:54 AM
I am wondering how it worked for you. I have tried the same steps and no
luck. using dbutils i am finding file but when i read using pandas its saying no such file or directory exists.
dbutils.fs.ls("FileStore/shared_uploads/kumarpalle/Covid19Europedata.csv")
Please help here. uploaded the same file that am working on.
Also, please check if you have applied any settings that we are missing.
08-18-2022 10:40 AM
Hi Kumar,
you're almost there, try this:
df1 = pd.read_csv("/dbfs/FileStore/shared_uploads/kumarpalle/Covid19Europedata.csv")
df1.describe()
Cheers!
08-19-2022 03:00 AM
Hi @Alexis ,
Thanks for quick response. It above method didn't worked but reading using spark and appending to Pandas it worked.
Here are the steps that i followed.
df1 = spark.read.format("csv").option("header", "true").load("dbfs:/FileStore/shared_uploads/kumarpalle/Covid19Europedata-1.csv").toPandas()
df1.head()
08-19-2022 03:01 AM
@Venky D Please follow the above steps to read using Spark as pandas doesn't work through dbfs.
03-03-2023 08:01 AM
same issue here... .toPandas() was also my only solution... otherwise error!
08-30-2023 08:46 AM
I did the same as you, but the reading does not work.
Can you help me? thanks
10-26-2021 02:21 AM
Please help me on this error
10-26-2021 02:36 AM
can you try without the dbfs part?
so /Filestore/tables/world_bank.csv
10-26-2021 03:46 AM
it's not working
03-04-2022 07:34 AM
Hi @Venky D , Is it working now?
Excited to expand your horizons with us? Click here to Register and begin your journey to success!
Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!