Hey Guys,

I'm having some permission issues using service principal and instance profile and i hope you could help me.

I created a service principal and attached to it an instance profile - databricks-my-profile.

I have a s3 bucket with policy that allow read/write only to service principal databricks-my-profile. this bucket has been mount into dbfs.

I have a cluster with databricks-my-profile instance profile.

While im able to read & write into this s3 bucket from databricks environment( from notebooks, jobs) which is good since the cluster have an instance profile that fits with the s3 bucket restrictions, I can't read & write data from this bucket using my service principal but i can see in its roles that databricks-my-profile exists for this specific sp.

I tried to copy files into the bucket using databricks cli and with the sp token and got an error.

Command use to upload files:

databricks fs ls dbfs:/mnt/my_mounted_bucket --profile my-service-principal

Error i get after runnnig the command:

Error: Authorization failed. Your token may be expired or lack the valid scope

Does some one have any idea why this is failing? or how i should debug this issue?

I check the s3 bucket policy and the restriction are only on instance profile - so this don't happening because ip restrictions or something like this.

Hope you can help me.

Thanks!