OSError: [Errno 7] Argument list too long

MUA
New Contributor

‎08-16-2023 12:23 PM

Getting this error in Databricks and don't know how to solve

OSError: [Errno 7] Argument list too long: '/dbfs/databricks/aaecz/dev/w000aaecz/etl-framework-adb/0.4.31-20230503.131701-1/etl_libraries/utils/datadog/restart_datadog.sh'

if anyone can help

 

0 Kudos

SparkJun
Databricks Employee
Databricks Employee

‎09-01-2023 02:39 PM

Hi MUA, can you please firstly validate this DBFS path by using %fs ls bash command to list out the objects in this path? Then, the error OSError: [Errno 7] Argument list too long occurs when the number of command line arguments or the total size of the command line arguments is larger than the system's limit. This is a system-level error, not specific to any particular programming language or application by Spark. If the arguments can't be reduced, consider storing these arguments in a file and modify your script or command to read from this file instead of taking arguments from the command line. 

args = json.loads(json_file_path)
print(args)

 

jose_gonzalez
Databricks Employee
Databricks Employee

‎09-08-2023 03:55 PM

@MUA 

Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

0 Kudos