topic Getting "java.io.FileNotFoundException" when I'm trying to use wildcard character in Data Engineering

Getting "java.io.FileNotFoundException" when I'm trying to use wildcard character

Varshith — Sat, 31 Dec 2022 05:32:56 GMT

Though there are parquet files in the folder which I'm trying to query, I'm getting FileNotFoundException. Please Let me know if there is any issue in the code.

Thanks

Re: Getting "java.io.FileNotFoundException" when I'm trying to use wildcard character

Aviral-Bhardwaj — Sun, 01 Jan 2023 08:02:12 GMT

@Varshith K the actual path is

/dbfs/user/hive/warehouse/a/*.parquet

/user/hive/warehouse/a/*.parquet

check both and I am pretty sure that 1st one will work

Please select this answer as best answer if it resolved your issue

Thanks

Aviral Bhardwaj

Re: Getting "java.io.FileNotFoundException" when I'm trying to use wildcard character

Varshith — Sun, 01 Jan 2023 08:31:52 GMT

@Aviral Bhardwaj

Hi Aviral,

Actually when I'm giving "dbfs:/user/hive/warehouse/a/" or "/user/hive/warehouse/a/" as path then there is no issue, but when I'm trying to use wildcard character to find the parquet files in that folder then it is throwing error. Please refer to the attached screenshot

Thanks,

Varshith

Re: Getting "java.io.FileNotFoundException" when I'm trying to use wildcard character

Aviral-Bhardwaj — Sun, 01 Jan 2023 09:22:53 GMT

Hey @Varshith K Connect with me at https://www.linkedin.com/in/aviralb/

we will set up a meeting link and I will try to help you

Re: Getting "java.io.FileNotFoundException" when I'm trying to use wildcard character

ramravi — Mon, 02 Jan 2023 14:02:14 GMT

alternatively you can use this as shell command. This will work.

%sh
 
ls /dbfs/mnt/mountpoint/path/to/raw/*.parquet