cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

No Longer Able to Create DeltaTables in ADLS Gen 2

Carlton
Contributor II

Hi Community,

Up until recently I was happily deleting deltaTables in ADLS Gen with their associated _delta_log table, and subsequently recreating the same table with a new _delta_log table.

Now, after deleting a table with its associated _delta_log table when I attempt to create a new table with the same name I get the error:

DeltaIllegalStateException: The protocol of your Delta table could not be recovered while Reconstructing version: 0. Did you manually delete files in the _delta_log directory?

Has Databricks changed something that prevents people from recreating new deltaTables with the same table name?

Can someone please let me know how to resolve this.

 

 

19 REPLIES 19

szymon_dybczak
Esteemed Contributor III

Thanks for additional details. Could you also tell us if you're using Hive metastore or Unity Catalog?

 

If you are using Hive then how you have configured storage?

Also, do you see something useful in driver logs?

Carlton
Contributor II

Hi @szymon_dybczak at the moment, I'm using Databricks Community Edition, however I get the same issue when I'm using a premium edition of Databricks

Carlton_0-1758125389872.png

 

Carlton
Contributor II

Any more thoughts on this issue guys

szymon_dybczak
Esteemed Contributor III

So, since you're using databricks community edition I'm assuming that you're dealing with Hive metastore.
You can try following:

1. Go to Settings 

szymon_dybczak_0-1758130627296.png

2. Now go to Advance (2) and turn on DBFS File Browser (3)

szymon_dybczak_1-1758130708447.png

 

3. Now, click Catalog (4), then DBFS(5) and go to users -> hive -> warehouse and check if you can find the table that you want to delete (Country).

4. If the table is there, copy path to the table (just like I did for flattened_source table in screen below)

szymon_dybczak_3-1758131344216.png

 

5. Then remove all files using dbutils.fm.rm

dbutils.fs.rm(dir="dbfs:/user/hive/warehouse/invoices_bz", recurse=True)

6. Double check that there are no files in your ADLS related to Country table

7. Try to recreate it.

Carlton
Contributor II

Hi @szymon_dybczak , you must have a special version of Databricks Community Edition as I don't have those options after I select settings

 

Carlton_1-1758132216163.png

 

 

Shall I try from my premium paid for version of Databricks?

 

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now