How do I download and unzip datasets from Kaggle into DBFS?

StephanieAlba
Databricks Employee
Databricks Employee

‎03-22-2023 07:30 AM

 
Labels:
0 Kudos

StephanieAlba
Databricks Employee
Databricks Employee

‎03-22-2023 07:32 AM

As shown here on StackOverflow

import opendatasets as od
 
od.download("https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/data","/dbfs/FileStore/mypath/")

 The output, when running this, shows first the zip being downloaded. Once the download is complete, it automatically extracts or unzips the files.

Extracting archive /dbfs/FileStore/mypath/tlvmc-parkinsons-freezing-gait-prediction/tlvmc-parkinsons-freezing-gait-prediction.zip to /dbfs/FileStore/mypath/tlvmc-parkinsons-freezing-gait-prediction
 

View solution in original post

pvignesh92
Honored Contributor
In response to StephanieAlba

‎03-22-2023 08:22 AM

Hi @Stephanie Rivera​. In the Databricks notebook, you can handle this with either Python, Scala or bash.

I have not tried it but below should work as this command works in native shell. 

%sh curl some_url --output myfile.zip
 
%sh unzip myfile.zip -d "some directory"

0 Kudos

karthik_p
Databricks Partner

‎03-22-2023 09:14 AM

@Stephanie Rivera​ please download u r kaggle file and unzip, if it is less than 100mb, you can follow below steps to directly to upload https://docs.databricks.com/ingestion/add-data/upload-data.html else please take below example and replace u r zip url and target

%sh curl https://resources.lendingclub.com/LoanStats3a.csv.zip --output /tmp/LoanStats3a.csv.zip

unzip /tmp/LoanStats3a.csv.zip

karthik.p
0 Kudos

Debayan
Databricks Employee
Databricks Employee

‎03-22-2023 10:43 PM

Hi, You can refer to https://docs.databricks.com/files/unzip-files.html. You can curl the file you want and then it can be unzipped as mentioned in the doc.

Please let us know if this helps.

Also, please tag @Debayan with your next update which will notify me.

0 Kudos