topic Re: Downloading and storing a PDF file to FileStore not working in Data Engineering

Downloading and storing a PDF file to FileStore not working

msa_2j212 — Thu, 29 Jun 2023 10:47:26 GMT

I'm trying to download a PDF file and store it in FileStore using this code in a Notebook:

with open('/dbfs/FileStore/file.pdf', 'wb') as f: f.write(requests.get('https://url.com/file.pdf').content)

But I'm getting this error:

FileNotFoundError: [Errno 2] No such file or directory

What am I doing wrong?

Brian2 — Thu, 29 Jun 2023 13:09:37 GMT

Might be easier to use curl commnad .. in a notebook you can run as shell command or python to first load the file into local driver temp storage

%sh curl https://url.com/file.pdf --output /tmp/file.pdf

or in python

import urllib
urllib.request.urlretrieve("https://url.com/file.pdf", "/tmp/file.pdf.csv")

Then move the file to DBFS

dbutils.fs.mv("file:/tmp/file.pdf", "dbfs:/Filestore/file.pdf

msa_2j212 — Thu, 29 Jun 2023 14:14:03 GMT

This worked, thanks.