The problem is that we shouldnt be using json keys at all if we are running databricks on GCP. with "No Isolation Shared" its able to query the gce metadata service and get the credentials of the service account attached to the instance. There is even a setting for selecting a service account when creating a cluster so it should be supported.