cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 
Help
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Search instead forย 
Did you mean:ย 

File size upload limit through CLI

sheilaL
New Contributor II

โ€Ž01-08-2024 01:45 PM

Does anyone know the size limit for uploading files through the CLI? I'm not finding it in the documentation.

0 Kudos
3 REPLIES 3

Kaniz_Fatma
Community Manager
Community Manager

โ€Ž01-08-2024 11:30 PM

Hi @sheilaL, For Databricks, the size limit for uploading files through the Command Line Interface (CLI) is 2GB. If you use local file I/O APIs to read or write files larger than 2GB, you might see corrupted files. Instead, for files larger than 2GB, it is recommended to use the DBFS CLI, dbutils, or Spark APIs.

0 Kudos

sheilaL
New Contributor II
In response to Kaniz_Fatma

โ€Ž01-10-2024 06:10 AM

Thanks, Kaniz. So I saw in the UI that the limit is 2GB. The limit is the same from the command line?

0 Kudos

User16413245720
New Contributor II
In response to Kaniz_Fatma

Thursday

the 2GB local file API limit no longer applies (not since DBR 6)

0 Kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group
Announcements