topic Re: hi all - I have created a data frame and would like to save in delta format using df.write.format("delta").saveAsTable("tablename") Its not working and throws " AnalysisException" . Please advise . in Data Engineering

hi all - I have created a data frame and would like to save in delta format using df.write.format("delta").saveAsTable("tablename") Its not working and throws " AnalysisException" . Please advise .

joakon — Thu, 17 Nov 2022 04:15:17 GMT

Pat — Thu, 17 Nov 2022 12:44:49 GMT

Hi @raghu maremanda ,

can you please share error details?

are you using PYTHON CELL to execute the code?

I am not sure right now if it's optional or mandatory, but you could add .mode() i.e:

df.write.format("delta").mode("append").saveAsTable("tablename")
 
append or overwrite

joakon — Thu, 17 Nov 2022 16:29:56 GMT

Thank you for your response.

This is what I have so far -

d = spark.read.option("inferSchema" ,"true").option("delimeter" ,",").option("header","true").csv('dbfs:/FileStore/tables/Devices.csv')

d.write.format("delta").mode("append").saveAsTable("devices_new")

Pat — Thu, 17 Nov 2022 21:50:59 GMT

@raghu maremanda ,

you didn't add the .mode() in your code shown.

it's still:

d.write.format("delta").saveAsTable("delta.devices_new")

I am not sure if this will help, but try:

d.write.format("delta").mode("append").saveAsTable("devices_new")

also, the error doesn't tell much. can you share the error thrown?

thanks,

Pat.

Anonymous — Mon, 21 Nov 2022 05:48:45 GMT

Your code is not wrong, I tried and executed from my side and no error happens.

Could you check version of DBR?

joakon — Tue, 29 Nov 2022 21:01:40 GMT

thank you for the response. db runtime was older. issue resolved.

huyd — Sat, 17 Dec 2022 14:28:59 GMT

check your read cell, "Delimeter"