topic Loading tables to gold, one loads and the other two fail but same process. in Data Engineering

Loading tables to gold, one loads and the other two fail but same process.

190809 — Mon, 06 Feb 2023 09:27:37 GMT

Hi team, I am still fairly new to working with delta tables. I have created a df by reading in data from existing silver tables in my lakehouse. I read in the silver tables usiung sql into a workbook, do some manipulation, unnest some fiels and then attempt to load as gold dlt tables. One of my gold tables works and the other two fail with the error 'cannot create ... because managed table already exists'. Any tips, tricks, advice on how to handle this. My dlt load looks like this:

<@dlt.table(comment='gold_users processed from silver platform users table to create gold tables ready for aggregation.'

)

def gold_users():

return users_df/>

Re: Loading tables to gold, one loads and the other two fail but same process.

ChaubeyPravin — Tue, 07 Feb 2023 11:40:31 GMT

Hi ,

It seems like there is already a managed table with same name.

If you want to anyhow write you need to remove first exist table with same name or if your schema is same you can append data to table.

To remove existing table you can use dbutils.fs.rm('path',True)

Re: Loading tables to gold, one loads and the other two fail but same process.

190809 — Tue, 07 Feb 2023 12:27:34 GMT

Hi @Pravin Chaubey thanks for responding. I discovered the issue. I had to load them as unmanaged tables but had previously not specified a path when doing .saveAsTable() and so those two tables that were failing to load were in fact managed tables as you said.