topic Re: Renamed table cannot be written to or deleted from in Data Engineering

Renamed table cannot be written to or deleted from

477061 — Fri, 26 Aug 2022 15:32:13 GMT

I have renamed a table, however on trying to write to it (or delete from it) I get the following error:

`java.io.FileNotFoundException: No such file or directory: s3a://.../hive/warehouse/testing.db/renamed_table_name/_delta_log/00000000000000000002.json`

My cluster version is 10.4 LTS

The following python snippet can be used to recreate the issue:

```

DB_NAME = 'testing'

def rename(old_table_name, new_table_name):

spark.conf.set("spark.databricks.delta.alterTable.rename.enabledOnAWS", "true")

spark.sql(f"ALTER TABLE {DB_NAME}.{old_table_name} RENAME TO {DB_NAME}.{new_table_name}")

def overwrite(table_name, source_df):

(source_df.write

.mode("overwrite")

.format("delta")

.saveAsTable(f"{DB_NAME}.{table_name}")

)

def get_df():

data = [

("Name1", "Surname1", 12345),

("Name2", "Surname2", 67890),

]

return spark.createDataFrame(data)

sparkDf = get_df()

# Create table

overwrite('table_name', sparkDf)

# Rename table

rename('table_name', 'renamed_table_name')

# Try to write to renamed table (or delete from). Failure happens here

overwrite('renamed_table_name', sparkDf)

```

Any help would be appreciated

Re: Renamed table cannot be written to or deleted from

AmanSehgal — Wed, 31 Aug 2022 13:16:26 GMT

I tried your code in 11.1 (includes Apache Spark 3.3.0, Scala 2.12) and it worked perfectly fine.

However it will work perfectly only the first time because if you try to rerun the same code, it'll error out stating renamed_table_name exists.

P.S: Please use code snippet(</> button) to include your code in your question

Re: Renamed table cannot be written to or deleted from

Vidula — Thu, 15 Sep 2022 10:36:03 GMT

Hi @julie.holtzhausen

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: Renamed table cannot be written to or deleted from

Noopur_Nigam — Fri, 30 Sep 2022 11:19:53 GMT

Hi @477061 Could you please try to test it in DBR 11.1 and see if the issue persists for you?