topic Re: Streaming xls files Using Auto Loader in Get Started Discussions

Streaming xls files Using Auto Loader

YS1 — Wed, 17 Jan 2024 21:15:08 GMT

Hello,

Is there a way to read .xls files using auto loader or is there any workaround since excel files are not supported by the auto loader per the following document?

https://docs.databricks.com/en/ingestion/auto-loader/options.html

Thanks.

Re: Streaming xls files Using Auto Loader

shan_chandra — Wed, 17 Jan 2024 22:27:12 GMT

@YS1 - Can you please check if you can convert .xls to .csv files and ingest using autoloader?

Re: Streaming xls files Using Auto Loader

YS1 — Wed, 17 Jan 2024 22:49:20 GMT

@Isha I tried to read them directly as csv files using different delimiters but the files weren't parsed correctly. The problem is that you have to check for the files that weren't read by the auto loader and then convert them to csv which is not the target here of the auto loader. My main goal is to read the files as they arrive to my folder without the need to check which one was read before.

Re: Streaming xls files Using Auto Loader

jose_gonzalez — Thu, 18 Jan 2024 00:21:03 GMT

You can try to read the file as text file, using "wholeText" option https://docs.databricks.com/en/ingestion/auto-loader/options.html#text-options and then the whole file contents will be passed to the from_xml function and it will return an array of the records. Give it a try.

Re: Streaming xls files Using Auto Loader

YS1 — Thu, 18 Jan 2024 16:00:52 GMT

@jose_gonzalez I tried that, but it's giving me weird values. not sure what's wrong with that.

Re: Streaming xls files Using Auto Loader

Sicnarf — Wed, 03 Jul 2024 17:45:50 GMT

I am facing the same issue--I have a stream that I'd like to use autoloader on with an .xlsx. Is there any update to any workarounds on this issue?

Re: Streaming xls files Using Auto Loader

YS1 — Fri, 05 Jul 2024 16:26:12 GMT

I couldn't find any. I had to change the type of the files.