cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks Workflow dbt-core job failure with Connection aborted

ChsAIkrishna
New Contributor III

When we are using dbt-core task on databricks workflow, each 100 workflow executions one job is failing with below reason after the reboot it works well what would be the permanent remediation ? 

('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

1 ACCEPTED SOLUTION

Accepted Solutions

Walter_C
Databricks Employee
Databricks Employee

Here are some steps you can take to address this issue:

  1. Update dbt-databricks Version: Ensure that you are using the latest version of dbt-databricks. The issue has been addressed in version 1.7.14, which includes changes to connection management and logging improvements.

  2. Adjust Connection Settings: Modify the connection settings in your dbt profile to reduce the likelihood of idle connections being closed. Specifically, you can set the connect_max_idle parameter to a lower value, such as 60 seconds. This setting ensures that connections are not idle for too long, which can help prevent them from being closed unexpectedly.

    Example configuration:

    connection_parameters:
      connect_max_idle: 60
  3. Increase Connection Retries: Increase the number of connection retries and the timeout settings to provide more resilience against transient connection issues.

    Example configuration:

connect_retries: 6
connect_timeout: 600
_socket_timeout: 1200

View solution in original post

2 REPLIES 2

Walter_C
Databricks Employee
Databricks Employee

Here are some steps you can take to address this issue:

  1. Update dbt-databricks Version: Ensure that you are using the latest version of dbt-databricks. The issue has been addressed in version 1.7.14, which includes changes to connection management and logging improvements.

  2. Adjust Connection Settings: Modify the connection settings in your dbt profile to reduce the likelihood of idle connections being closed. Specifically, you can set the connect_max_idle parameter to a lower value, such as 60 seconds. This setting ensures that connections are not idle for too long, which can help prevent them from being closed unexpectedly.

    Example configuration:

    connection_parameters:
      connect_max_idle: 60
  3. Increase Connection Retries: Increase the number of connection retries and the timeout settings to provide more resilience against transient connection issues.

    Example configuration:

connect_retries: 6
connect_timeout: 600
_socket_timeout: 1200

ChsAIkrishna
New Contributor III

@Walter_C  Kudo's to you, Thank you very much, we placed the "connect retries" lets see. 
Ref : https://docs.getdbt.com/docs/core/connect-data-platform/databricks-setup#additional-parameters

 

 

 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group