Using F Strings in filenames to read in files from mounted containers
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-29-2023 07:24 AM
Below are 2 versions of codes I tried and the failure messages. Does anyone have advice how to form this properly?
I tried this
for x in range(2,num_organization_page,1):
file_str = f"dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization{x}.json",
print(file_str)
organizations_df = spark.read.option("multiline","true").json(f"{file_str}")
and got this error:
('dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization2.json',)
IllegalArgumentException: java.net.URISyntaxException: Illegal character in scheme name at index 0: ('dbfs:/mnt/dlsi2023/raw/ds_pet_finder_organization2.json',)
I tried this
for x in range(2,num_organization_page,1):
file_str = f"/mnt/dlsi2023/raw/ds_pet_finder_organization{x}.json",
print(file_str)
organizations_df = spark.read.option("multiline","true").json(f"{file_str}")and got this error:
('/mnt/dlsi2023/raw/ds_pet_finder_organization2.json',)
IllegalArgumentException: Path must be absolute: ('/mnt/dlsi2023/raw/ds_pet_finder_organization2.json',)