topic Re: [TABLE_OR_VIEW_ALREADY_EXISTS] when running create or replace view in Get Started Discussions

[TABLE_OR_VIEW_ALREADY_EXISTS] when running create or replace view

claudiazi — Wed, 05 Jun 2024 17:24:23 GMT

When I'm running `dbt run -s model` for different models in parallel on databricks general compute cluster. Im getting the error: TABLE_OR_VIEW_ALREADY_EXISTS. Also, at the same time, the view/table was not created at all.

But I run them in sequence, then no such error will show.

Any idea what is causing this?

Re: [TABLE_OR_VIEW_ALREADY_EXISTS] when running create or replace view

raphaelblg — Thu, 06 Jun 2024 19:54:45 GMT

Hi @claudiazi,

It's hard to determine the exact root cause of the issue, but there's a known issue that could lead to the same error.

If you check CREATE TABLE [USING] - Parameters. You'll be able to see the following note:

Databricks strongly recommends using REPLACE instead of dropping and re-creating Delta Lake tables.

This is because the pattern of deleting and creating Delta Lake tables without the REPLACE clause might not clean-up all the metadata files right away.

Re: [TABLE_OR_VIEW_ALREADY_EXISTS] when running create or replace view

claudiazi — Fri, 07 Jun 2024 13:34:45 GMT

@raphaelblg thaanks a lot for your reply! However the command were `create or replace view/table` Thats why I'm very surprised that we are getting this error message 🤔🤔🤔

Re: [TABLE_OR_VIEW_ALREADY_EXISTS] when running create or replace view

raphaelblg — Fri, 07 Jun 2024 14:10:23 GMT

@claudiazi Are you the owner of the view?

Re: [TABLE_OR_VIEW_ALREADY_EXISTS] when running create or replace view

claudiazi — Fri, 07 Jun 2024 14:44:36 GMT

@raphaelblg yes! Im the owner. However, these views are inside the hive_metastore catalog. Could it be the reason?
Many thaaanks!

Re: [TABLE_OR_VIEW_ALREADY_EXISTS] when running create or replace view

raphaelblg — Fri, 07 Jun 2024 15:06:46 GMT

@claudiazi from what we've discussed so far I see no reason for the failure.

Not sure if this error is something the community can help with but maybe opening a support ticket through https://help.databricks.com/s/ might be a good idea.

Where's this query being submitted? Interactive cluster, SQL Warehouse, dbt?