cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

After catalog recreation I cannot create folders with names that previously had existed

JakubMlacki
New Contributor III

I have a catalog with multiple schemas, tables and volumes. Each volume contains two folders with specific names that I cannot change.

When I drop the whole catalog (CASCADE) and create catalog with the same name, with the same schemas, tables and volumes, inside volumes I cannot create those folders neither spark nor dbutils. It works via UI though... I get error:

Failed to acquire a SAS token for create-directory on /my_catalog/__unitystorage/catalogs/35a66a07-e1d4-4f0a-931a-8ba1d0f4578e/volumes/4ae1832e-f9c0-470d-9c06-80fc7a290849/src due to java.util.concurrent.ExecutionException: com.databricks.sql.managedcatalog.UnityCatalogServiceException: [RequestId=8e073a68-fd74-43c7-a57b-a0879480d0c2 ErrorClass=SCHEMA_DOES_NOT_EXIST.RESOURCE_DOES_NOT_EXIST] Schema '1db0471a-0e8d-450c-bb3b-bfd216ef26ad' does not exist.

those IDs refers to the previously dropped catalog, not the current one.

After 24 hours the same command succeeds. Is there any metadata caching? Can I clean it manually?

1 REPLY 1

-werners-
Esteemed Contributor III

I also had issues a while back with (what I believe is) some kind of cache.  I have no knowledge how to override this.
What I did is what you tried: waiting.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group