Hi @Ellaine Ho​ ,

In such cases like yours , key file can be encoded to base64. There are multiple methods to work with base64 encoded keys in databricks.

If you are uploading it in dbfs then you can encode it with base64 encoder online (https://www.base64encode.org/) upload it in DBFS and mention the path.

The best practice can be using secrets. You can use the above tool to Base64-encode the contents of your JSON key file, create a secret in a Databricks-backed scope and then you can copy & paste the Base64-encoded text into your secret value. After that, you can reference your secret with the following Spark config of your cluster:

spark.conf.set("credentials", base64_string)

If it is for any configuration in cluster:

You can encode private key file with base64 encoder online (https://www.base64encode.org/) and mention the below config in spark config. (for base64 encoding, you have to encode the entire file, example below)

{
  "type": "service_account",
  "project_id": "anything",
  "private_key_id": "***",
  "private_key": "-----BEGIN PRIVATE KEY-----\nngkzh\nVWx+z/ISmhN5x9xLINbU5IA+anbH0/lbw5s=\n-----END PRIVATE KEY-----\n",
  "client_email": "***@xyz.com",
  "client_id": "19208260",
  "auth_uri": "https://accounts.googleapis.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/cloud-sql-proxy%40fe-dev-sandbox.iam.com"
}

Please let us know if you need further clarification on the same. We are more than happy to assist you further.