topic Re: Databricks Workflow dbt-core job failure with Connection aborted in Data Engineering

Databricks Workflow dbt-core job failure with Connection aborted

ChsAIkrishna — Thu, 14 Nov 2024 11:26:02 GMT

When we are using dbt-core task on databricks workflow, each 100 workflow executions one job is failing with below reason after the reboot it works well what would be the permanent remediation ?

('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Re: Databricks Workflow dbt-core job failure with Connection aborted

Walter_C — Thu, 14 Nov 2024 14:29:55 GMT

Here are some steps you can take to address this issue:

Update dbt-databricks Version: Ensure that you are using the latest version of dbt-databricks. The issue has been addressed in version 1.7.14, which includes changes to connection management and logging improvements.
Adjust Connection Settings: Modify the connection settings in your dbt profile to reduce the likelihood of idle connections being closed. Specifically, you can set the connect_max_idle parameter to a lower value, such as 60 seconds. This setting ensures that connections are not idle for too long, which can help prevent them from being closed unexpectedly.

Example configuration:

connection_parameters: connect_max_idle: 60
Increase Connection Retries: Increase the number of connection retries and the timeout settings to provide more resilience against transient connection issues.

Example configuration:

connect_retries: 6 connect_timeout: 600 _socket_timeout: 1200

Re: Databricks Workflow dbt-core job failure with Connection aborted

ChsAIkrishna — Fri, 15 Nov 2024 13:39:20 GMT

@Walter_C Kudo's to you, Thank you very much, we placed the "connect retries" lets see.
Ref : https://docs.getdbt.com/docs/core/connect-data-platform/databricks-setup#additional-parameters