Ok, so it seems that you're configuring things correctly. So maybe look at below thread, @jmbabazi  has similiar issue with

JDBC ERROR: [Databricks][JDBCDriver](500151) Error setting/closing session: Invalid local Address 

It seems that this can be known bug and team is preparing fix for it. At least that's what below thread says:

Solved: Re: Unable to connect to SQL Warehouse from RStudi... - Databricks Community - 83498