https://community.databricks.com/t5/data-engineering/overwriting-same-table/m-p/74010#M34697 <P>Found this to be a transient error. Once i restarted the cluster, the overwrite was successful.&nbsp;</P> Fri, 14 Jun 2024 11:00:05 GMT naga_databricks 2024-06-14T11:00:05Z https://community.databricks.com/t5/data-engineering/overwriting-same-table/m-p/74005#M34696 <P>I have a table A that is used in a spark.sql and joins with multiple other tables to get data. this data will be overwritten to the same table A.</P><P>When i tried this, i get an error consistently as below:&nbsp;</P><PRE><FONT face="courier new,courier"><SPAN>ERROR: An error occurred while calling o382.saveAsTable. Trace: py4j.Py4JException: Method saveAsTable([class java.util.HashSet]) does not exist at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:344) at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:352) at py4j.Gateway.invoke(Gateway.java:297) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:199) at py4j.ClientServerConnection.run(ClientServerConnection.java:119) at java.lang.Thread.run(Thread.java:750) </SPAN></FONT></PRE><P>I tried writing same dataframe into new table and it works fine.</P><P>Are there some considerations to make when a dataframe containing results of a table is overwritten to same table?</P> Fri, 14 Jun 2024 10:50:32 GMT https://community.databricks.com/t5/data-engineering/overwriting-same-table/m-p/74005#M34696 naga_databricks 2024-06-14T10:50:32Z https://community.databricks.com/t5/data-engineering/overwriting-same-table/m-p/74010#M34697 <P>Found this to be a transient error. Once i restarted the cluster, the overwrite was successful.&nbsp;</P> Fri, 14 Jun 2024 11:00:05 GMT https://community.databricks.com/t5/data-engineering/overwriting-same-table/m-p/74010#M34697 naga_databricks 2024-06-14T11:00:05Z