External table from external location

210227 — Fri, 17 May 2024 09:10:32 GMT

Hi,

I'm creating external table from existing external location and am a bit puzzled as to what permissions I need for it or what is the correct way of defining the S3 path with wildcards.

This:

create external table if not exists test_catalogue_dev.bronze.test_external

using csv

location 's3://test-data/full/2023/01/*/'

OPTIONS(

recursiveFileLookup "true",

ignoreExtension "true",

infereSchema "true"

)
returns java.nio.file.AccessDeniedException, yet:

create external table if not exists test_catalogue_dev.bronze.test_external

using csv

location 's3://test-data/full/2023/01/01/'

OPTIONS(

recursiveFileLookup "true",

ignoreExtension "true",

infereSchema "true"

)

happily creates one.

Is using a wildcard trying to put something in the external bucket?

Re: External table from external location

210227 — Fri, 17 May 2024 09:51:06 GMT

Just for the reference, the wildcard is not needed in this case, just a misleading error message. In this case 's3://test-data/full/2023/01/' instead of 's3://test-data/full/2023/01/*/' was the correct PATH

topic External table from external location in Data Engineering

External table from external location

Re: External table from external location