Performance issues when loading an Excel file from DBFS using R

[mbaumga]

I have uploaded small Excel files on my DBFS. I then use function read_xlsx() from the "readxl" package in R to import the file into the R memory. I use a standard cluster (12.1, non ML). The function works but it takes ages. E.g. a simple Excel table with 40000+ records and 5 columns takes 9 minutes. On my R installation on Windows, the load is instantaneous. "readxl" is considered to be the best package to deal with Excel files. It is part of the already made available libraries on the cluster. Any idea what might cause this?

[mbaumga]

Dear Kaniz, thanks. Your suggestion to copy the Excel file from DBFS to a temp folder "e.g. /tmp/" worked. Once I run read_xlsx() the upload is instantenous.

Indeed, we had the same problem the other way around. I was not able to save an xlsx or pdf file into a folder on "/dbfs/FileStore/*". However, I can save it to "/tmp/*" and then copy it to where I want the file to be. Good to know!

Thanks again!

[Anonymous]

Hi @Marcel Baumgartner​ 

Thank you for posting your question in our community! We are happy to assist you.

To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?

This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance! 

[mbaumga]

The questions was answered, successfully. All okay!