cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Loading tables to gold, one loads and the other two fail but same process.

190809
Contributor

Hi team, I am still fairly new to working with delta tables. I have created a df by reading in data from existing silver tables in my lakehouse. I read in the silver tables usiung sql into a workbook, do some manipulation, unnest some fiels and then attempt to load as gold dlt tables. One of my gold tables works and the other two fail with the error 'cannot create ... because managed table already exists'. Any tips, tricks, advice on how to handle this. My dlt load looks like this:

<@dlt.table(comment='gold_users processed from silver platform users table to create gold tables ready for aggregation.'

)

def gold_users():

return users_df/>

1 ACCEPTED SOLUTION

Accepted Solutions

190809
Contributor

Hi @Pravin Chaubey​ thanks for responding. I discovered the issue. I had to load them as unmanaged tables but had previously not specified a path when doing .saveAsTable() and so those two tables that were failing to load were in fact managed tables as you said.

View solution in original post

2 REPLIES 2

ChaubeyPravin
New Contributor II

Hi ,

It seems like there is already a managed table with same name.

If you want to anyhow write you need to remove first exist table with same name or if your schema is same you can append data to table.

To remove existing table you can use dbutils.fs.rm('path',True)

190809
Contributor

Hi @Pravin Chaubey​ thanks for responding. I discovered the issue. I had to load them as unmanaged tables but had previously not specified a path when doing .saveAsTable() and so those two tables that were failing to load were in fact managed tables as you said.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.