cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
cancel
Showing results for 
Search instead for 
Did you mean: 

Using F Strings in filenames to read in files from mounted containers

jch
New Contributor III

Below are 2 versions of codes I tried and the failure messages. Does anyone have advice how to form this properly?

I tried this

for x in range(2,num_organization_page,1):
file_str = f"dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization{x}.json",
print(file_str)
organizations_df = spark.read.option("multiline","true").json(f"{file_str}")

and got this error:
('dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization2.json',)
IllegalArgumentException: java.net.URISyntaxException: Illegal character in scheme name at index 0: ('dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization2.json',)

I tried this

for x in range(2,num_organization_page,1):
file_str = f"/mnt/dlsi2023/raw/ds_pet_finder_organization{x}.json",
print(file_str)
organizations_df = spark.read.option("multiline","true").json(f"{file_str}")

and got this error:

('/mnt/dlsi2023/raw/ds_pet_finder_organization2.json',)
IllegalArgumentException: Path must be absolute: ('/mnt/dlsi2023/raw/ds_pet_finder_organization2.json',)

 

1 REPLY 1

erigaud
Honored Contributor

Your code works, you just have to remove the comma at the end of the second line

file_str = f"dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization{x}.json",

 Should be 

file_str = f"dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization{x}.json"

 This is what is causing the error.

Also note that 

f"{file_str}" is equivalent to just file_str : No f-string needed here !