Upgrading to UC. Parent external location for path `s3://____` does not exist"...but it does?

colinsorensen — Sat, 16 Mar 2024 05:06:11 GMT

Topic. I am trying to upgrade some external tables in our hive metastore to the unity catalog. I used the upgrade functionality in the UI, as well as its provided SQL:

CREATE TABLE `unity_catalog`.`default`.`table` LIKE `hive_metastore`.`schema`.`table` COPY LOCATION;

ALTER TABLE `hive_metastore`.`schema`.`table` SET TBLPROPERTIES ('upgraded_to' = '`unity_catalog`.`default`.`table`', 'comment' = 'This table is deprecated. Please update to use `unity_catalog`.`default`.`table` instead of `hive_metastore`.`schema`.`table`');

But I get this error:

[RequestId=fe8c176e-a7d4-4639-bd7f-83fafca14131 ErrorClass=EXTERNAL_LOCATION_DOES_NOT_EXIST] parent external location for path `s3://_______` does not exist."

But it most definitely exists + the s3 bucket is mounted, so I'm not sure what it's even looking for the raw s3 location. Shouldn't it be looking for dbfs://mnt/________??

Would love any suggestions, thank you!

Re: Upgrading to UC. Parent external location for path `s3://____` does not exist"...but it doe

colinsorensen — Sat, 16 Mar 2024 05:23:15 GMT

When I have tried to edit the location to include the dbfs component (

CREATE TABLE`unity_catalog`.`default`.`table`LIKE`hive_metastore`.`schema`.`table` LOCATION 'dbfs:/mnt/foobarbaz')

I get a new error:

"[UPGRADE_NOT_SUPPORTED.UNSUPPORTED_FILE_SCHEME] Table is not eligible for upgrade from Hive Metastore to Unity Catalog. Reason: Unsupported file system scheme dbfs."

Similar error when you take out the dbfs stuff. Thank you!

topic Upgrading to UC. Parent external location for path `s3://____` does not exist"...but it does? in Data Engineering

Upgrading to UC. Parent external location for path `s3://____` does not exist"...but it does?

Re: Upgrading to UC. Parent external location for path `s3://____` does not exist"...but it doe