After catalog recreation I cannot create folders with names that previously had existed
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-20-2024 06:59 AM
I have a catalog with multiple schemas, tables and volumes. Each volume contains two folders with specific names that I cannot change.
When I drop the whole catalog (CASCADE) and create catalog with the same name, with the same schemas, tables and volumes, inside volumes I cannot create those folders neither spark nor dbutils. It works via UI though... I get error:
Failed to acquire a SAS token for create-directory on /my_catalog/__unitystorage/catalogs/35a66a07-e1d4-4f0a-931a-8ba1d0f4578e/volumes/4ae1832e-f9c0-470d-9c06-80fc7a290849/src due to java.util.concurrent.ExecutionException: com.databricks.sql.managedcatalog.UnityCatalogServiceException: [RequestId=8e073a68-fd74-43c7-a57b-a0879480d0c2 ErrorClass=SCHEMA_DOES_NOT_EXIST.RESOURCE_DOES_NOT_EXIST] Schema '1db0471a-0e8d-450c-bb3b-bfd216ef26ad' does not exist.those IDs refers to the previously dropped catalog, not the current one.
After 24 hours the same command succeeds. Is there any metadata caching? Can I clean it manually?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-21-2024 06:27 AM
I also had issues a while back with (what I believe is) some kind of cache. I have no knowledge how to override this.
What I did is what you tried: waiting.
