COPY INTO size limit

[DBUser2]

Hi

I'm using the COPY INTO command to ingest data into a delta table in my Azure Databricks instance. Sometime I get a timeout error running this command. Is there a limit on the size of the data that can be ingested using "COPY INTO" or limit on the number of files that can be ingested at a time?

Thanks

[Alberto_Umana]

Hello @DBUser2,

The COPY INTO command does not have a specific documented limit on the size of the data or the number of files that can be ingested at a time. Timeout errors can occur due to network issues, resource limitations, or long-running operations. Are you running the commands on a warehouse or over notebook?

Looking at cluster/warehouse metrics would be a good way to start investigating.

Also, do you have the statementID is executed via warehouse?

[DBUser2]

Thanks for replying, i'm running the command on sql warehouse.