Read the file from datalake in databricks -No such file directory error
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-20-2023 06:02 AM
0
I have a problem with reading the file from ADLS gen 2.
I have dont the mounting properly as after executing
dbutils.fs.ls('/mnt/bronze')
I can see the file path.
the way how I did the mounting:
# dbutils.fs.mount(
# source = "abfss://"+container_read+"@"+storage_account_name_read+".dfs.core.windows.net/",
# mount_point = "/mnt/dataverse",
# extra_configs = configs)
the way how I read the file:
the way how I read the file:
with open('/mnt/dataverse/model.json', 'r') as f:
data = f.read()
manifest = json.loads(data)and it throws the error
No such file or directory
Sketchy part is that I can read the file usign different cluster (
runtime 11.3.x-scala2.12
) but after switching to
12.2
cluster I cant read it.
Any idea how can I fix this?
- Labels:
-
File Path
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-20-2023 08:24 PM
Hi @Givi Salu
Great to meet you, and thanks for your question!
Let's see if your peers in the community have an answer to your question. Thanks.