cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

OSError: [Errno 7] Argument list too long

MUA
New Contributor

Getting this error in Databricks and don't know how to solve

OSError: [Errno 7] Argument list too long: '/dbfs/databricks/aaecz/dev/w000aaecz/etl-framework-adb/0.4.31-20230503.131701-1/etl_libraries/utils/datadog/restart_datadog.sh'

if anyone can help

 

2 REPLIES 2

SparkJun
Databricks Employee
Databricks Employee

Hi MUA, can you please firstly validate this DBFS path by using %fs ls bash command to list out the objects in this path? Then, the error OSError: [Errno 7] Argument list too long occurs when the number of command line arguments or the total size of the command line arguments is larger than the system's limit. This is a system-level error, not specific to any particular programming language or application by Spark. If the arguments can't be reduced, consider storing these arguments in a file and modify your script or command to read from this file instead of taking arguments from the command line. 

args = json.loads(json_file_path)
print(args)

 

jose_gonzalez
Databricks Employee
Databricks Employee

@MUA 

Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group