cancel
Showing results forĀ 
Search instead forĀ 
Did you mean:Ā 
Data Governance
Join discussions on data governance practices, compliance, and security within the Databricks Community. Exchange strategies and insights to ensure data integrity and regulatory compliance.
cancel
Showing results forĀ 
Search instead forĀ 
Did you mean:Ā 

Unity Catalog - Limited Options for Connection Objects

MiroFuoli
New Contributor II

Iā€™m currently trying to create a Foreign Catalog based on a Connection object of type SQLSERVER. This would allow me to directly access our on-premises MS SQL database from within Azure Databricks using Unity Catalog.

As Iā€™m part of a large organization, I have only limited control over the allowed connection types to this on-premises database. Thus, I'm required to specify the authentication scheme and enable integrated security to establish a successful connection.

Unfortunately, only a limited number of connection options are available according to the documentation, supported options are:

  • host
  • port
  • trustServerCertificate
  • user
  • password

When I connect to the on-premises database using PySpark with the SQL Server JDBC driver it works, as the driver allows me to pass a connection string where I can specify integrated security to be true and the correct authentication scheme. However, with this approach I can't leverage from federated queries and need to connect to each table in the foreign database separately.

Is there any other approach to creating a Connection object in Unity Catalog that allows me to pass additional configuration options? 

Ultimately, I would like to have my on-premises SQL Server database accessible within Unity Catalog so I can leverage federated query execution.

1 REPLY 1

Hi @Kaniz_Fatma,

Thank you for your answer. My question is not related to dbt, but rather Unity Catalog.

When creating a Connection object in Unity Catalog the default authentication method is Basic Authentication. I would like to know if I can override this setting. Specifically, I would like to know how I can specify Integrated Authentication when creating said Connection object in Unity Catalog?

Note: Nowadays, most corporate on-premises database systems reject connection attempts using Basic Authentication, as it is considered insecure. Integrated Authentication, however, relies on Kerberos technology, with strong credential encryption.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonā€™t want to miss the chance to attend and share knowledge.

If there isnā€™t a group near you, start one and help create a community that brings people together.

Request a New Group