topic Re: Getting access token error when connecting from azure databricks to GCS bucket in Get Started Discussions

Getting access token error when connecting from azure databricks to GCS bucket

kiko_roy — Fri, 01 Dec 2023 07:08:14 GMT

I am creating a data frame by reading a table's data residing in Azure backed unity catalog. I need to write the dataframe or file to GCS bucket. I have configured the spark cluster config using the GCP service account json values. Also tried uploading it to dbfs and accessed the same through .config ()

on running :

df1.write.format("parquet").save("gs://dev-XXXX-analyt-XXXXXXXX") getting error :
Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token
what could be the reason or resolution. Need help

Re: Getting access token error when connecting from azure databricks to GCS bucket

KristiLogos — Tue, 11 Feb 2025 23:30:10 GMT

DId you find out why this was happening? I'm having the same issue

Re: Getting access token error when connecting from azure databricks to GCS bucket

kiko_roy — Wed, 12 Feb 2025 08:34:52 GMT

@KristiLogos - Use the cluster access mode as : Single User and not shared and try

Re: Getting access token error when connecting from azure databricks to GCS bucket

KristiLogos — Wed, 12 Feb 2025 16:24:05 GMT

@kiko_roy unfortunately that didnt' work. the error is stating its trying to get the access token from metadata server, I wonder why from the metadata server?