Failing to install a library from dbfs mounted storage (adls2) with pass through credentials cluster

alonisser
Contributor II

We've setup a premium workspace with passthrough credentials cluster , while they do work and access my adls gen 2 storage

I can't make it install a library on the cluster from there. and keeping getting

"Library installation attempted on the driver node of cluster 0522-200212-mib0srv0 and failed. Please refer to the following error message to fix the library or contact Databricks support. Error Code: DRIVER_LIBRARY_INSTALLATION_FAILURE. Error Message: com.google.common.util.concurrent.UncheckedExecutionException: com.databricks.backend.daemon.data.client.adl.AzureCredentialNotFoundException: Could not find ADLS Gen2 Token

"

How can I actually install a "cluster wide" library? on those passthrough credentials clusters

(The general adls mount is using those credentials to mount the data lake)

This happens on both standard and high concurrency clusters