After catalog recreation I cannot create folders with names that previously had existed

JakubMlacki — Tue, 20 Aug 2024 13:59:04 GMT

I have a catalog with multiple schemas, tables and volumes. Each volume contains two folders with specific names that I cannot change.

When I drop the whole catalog (CASCADE) and create catalog with the same name, with the same schemas, tables and volumes, inside volumes I cannot create those folders neither spark nor dbutils. It works via UI though... I get error:

Failed to acquire a SAS token for create-directory on /my_catalog/__unitystorage/catalogs/35a66a07-e1d4-4f0a-931a-8ba1d0f4578e/volumes/4ae1832e-f9c0-470d-9c06-80fc7a290849/src due to java.util.concurrent.ExecutionException: com.databricks.sql.managedcatalog.UnityCatalogServiceException: [RequestId=8e073a68-fd74-43c7-a57b-a0879480d0c2 ErrorClass=SCHEMA_DOES_NOT_EXIST.RESOURCE_DOES_NOT_EXIST] Schema '1db0471a-0e8d-450c-bb3b-bfd216ef26ad' does not exist.

those IDs refers to the previously dropped catalog, not the current one.

After 24 hours the same command succeeds. Is there any metadata caching? Can I clean it manually?

Re: After catalog recreation I cannot create folders with names that previously had existed

-werners- — Wed, 21 Aug 2024 13:27:40 GMT

I also had issues a while back with (what I believe is) some kind of cache. I have no knowledge how to override this.
What I did is what you tried: waiting.

topic Re: After catalog recreation I cannot create folders with names that previously had existed in Data Engineering

After catalog recreation I cannot create folders with names that previously had existed

Re: After catalog recreation I cannot create folders with names that previously had existed