Lakehouse Federation - Unable to connect to Snowflake using "PEM Private Key"

[saiV06]

Hi,

I'm currently using Lakehouse Federation feature on databricks to run queries against Snowflake datawarehouse. Today I'm using a service credential to establish the connection (user id & pwd), but I have to change it to use private key. I tried using the "PEM Private Key" option, but it would never let me make the connection. Although, when I try to use the same key value from python it works. Attached the screenshot of the error message I'm getting. Can someone please assist on this?

Thanks!

[mark_ott]

To assist with your Databricks Lakehouse Federation to Snowflake using a PEM Private Key, let's clarify the underlying issue. You mentioned that:

• The connection works with a service credential (user id & password) but fails when switching to the "PEM Private Key" option in Databricks.

• The same PEM key does work from Python, but not from the Databricks interface.

• You have a screenshot of the error (but text description would be required for analysis as images can't be referenced directly).

Common PEM Key Connection Issues

• Key Format: Databricks typically requires the private key to be in PKCS#8 format. Many Python libraries will accept PKCS#1 or PKCS#8, but Databricks may only support PKCS#8 (-----BEGIN PRIVATE KEY-----). Using PKCS#1 (-----BEGIN RSA PRIVATE KEY-----) can result in connection errors.

• Key Encryption: Make sure the private key is not encrypted with a passphrase unless Databricks allows you to specify it. If your key is encrypted, it must be converted to an unencrypted version or provide the passphrase where supported.

• Line Endings: If pasting the key into Databricks configuration, ensure that line breaks are preserved and no extra spaces are added. Some interfaces strip or mess with line formatting.

• Permissions: Ensure the Snowflake user associated with the private key has the required roles and permissions for federation access.

Troubleshooting Steps

• Check Key Format: Convert your key to PKCS#8 using OpenSSL if required:

  bash

  openssl pkcs8 -topk8 -inform PEM -outform PEM -in private_key.pem -out pkcs8_key.pem -nocrypt

• Check Key Encoding: Use base64 encoding and ensure there is no trailing whitespace.

• Check Databricks Configuration: Make sure your Databricks connection configuration exactly matches Snowflake requirements, especially for account, user, and private_key fields.

• Compare with Python: If your Python environment works, compare how you're loading the key (is it unencrypted, in PKCS#8 format)? Replicate this exactly for Databricks.

Error Message Analysis

If you can provide the exact text of the error message from the screenshot, it would help address specific issues (for example, "unsupported key format" or "authentication failed"). Error strings often guide to the mismatched format or a missing detail.