We upgraded the GKE cluster to GCE cluster as per the databricks documentation. It works fine on GCE all purpose cluster but gives error while trying to access the databricks managed secret on GCE job cluster. Job is being executed using service principal and it has all the permission as it was working fine on GKE job cluster. Here is the error trace, 

: org.apache.http.conn.HttpHostConnectException: Connect to europe-west3.gcp.databricks.com:443 [europe-west3.gcp.databricks.com/34.159.208.230] failed: Connection timed out (Connection timed out) File <command-1511365916692010>, line 1 ----> 1 mongo_prd_user = dbutils.secrets.get(scope="<scope_name>", key="prd_user") 2 mongo_prd_password = dbutils.secrets.get(scope="<scope_name>",key="prd_password")