Accessing Excel file from Databricks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-02-2023 11:49 AM
Hi,
I am trying to access excel file that is stored in Azure Blob storage via Databricks.
In my understanding, it is not possible to access using Pyspark. So accessing through Pandas is the option,
Here is my code.
%pip install openpyxl
import pandas as pd
from pandas import ExcelFile
pdf = pd.ExcelFile('/mnt/dl-sy-team-dev/AD/AbsentEmployees.xls/', engine='openpyxl')
print(pdf)
ERROR
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/dl-sy-team-dev/ADP/AbsentEmployees.xls/'
The file is definitely there and has proper permissions as I can browse using dbutils.fs.ls command in databricks.
Can some one please help me with this error as to how to fix this?
Thanks,
John
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-03-2023 07:48 AM
Thank you Kaniz for the response.
1. Completed
2. This is completed as part of our initial environment setup OR is this something needs to be defined before the code below?
3. import pandas as pd
xls = pd.ExcelFile('wasbs://mnt/dl-sync-team-dev/ADP/AbsentEmployees.xls/') -- Accessing the excel file in Blob container using wasbs
network_master = pd.read_excel(xls, 'Sheet1')
Error : ValueError: Protocol not known: wasbs
Can you please take a look and correct me if there is anything wrong with my code or is there anything we need to configure in the environment for wasbs protocol?
Thanks,
John