- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-05-2025 01:20 PM
hi @der
Thank you for reaching out with your question, very interesting behavior.
I attempted to reproduce your issue using DBR 17.3 LTS but was unable to install the specific library version “dev.mauch:spark-excel_2.13:4.0.0_0.31.2.” However, I installed the next available version, “dev.mauch:spark-excel_2.13:4.0.0_0.32.2-prerelease0.”
With this version, I was able to use the “spark.read.format(“excel”)” function without any issue.
spark.read.format("excel").load(excel_file)
Aside from the links you shared, I couldn’t find documentation explaining why the error occurred with the “Excel” format. It appears that the issue may have been resolved in this newer version, possibly due to a missing internal registration.
I recommend updating your dev.mauch:spark-excel library to the newer version.
Alternatively, you can handle Excel (xlsx) files using the Pandas API on Spark, with the read_excel function, then convert it to a spark dataframe using the code below; just ensure you have “openpyxl” installed (using pip install)
import pyspark.pandas as ps
df = ps.read_excel(excel_file)
sp_df = df.to_spark()
display(sp_df)
I hope this helps and if it does, please Accept as Solution.
Thank you!