Hey Guys,
I'm having some permission issues using service principal and instance profile and i hope you could help me.
I created a service principal and attached to it an instance profile - databricks-my-profile.
I have a s3 bucket with policy that allow read/write only to service principal databricks-my-profile. this bucket has been mount into dbfs.
I have a cluster with databricks-my-profile instance profile.
While im able to read & write into this s3 bucket from databricks environment( from notebooks, jobs) which is good since the cluster have an instance profile that fits with the s3 bucket restrictions, I can't read & write data from this bucket using my service principal but i can see in its roles that databricks-my-profile exists for this specific sp.
I tried to copy files into the bucket using databricks cli and with the sp token and got an error.
Command use to upload files:
databricks fs ls dbfs:/mnt/my_mounted_bucket --profile my-service-principal
Error i get after runnnig the command:
Error: Authorization failed. Your token may be expired or lack the valid scope
Does some one have any idea why this is failing? or how i should debug this issue?
I check the s3 bucket policy and the restriction are only on instance profile - so this don't happening because ip restrictions or something like this.
Hope you can help me.
Thanks!
