Getting access token error when connecting from azure databricks to GCS bucket
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-30-2023 11:08 PM
I am creating a data frame by reading a table's data residing in Azure backed unity catalog. I need to write the dataframe or file to GCS bucket. I have configured the spark cluster config using the GCP service account json values. Also tried uploading it to dbfs and accessed the same through .config ()
on running :
df1.write.format("parquet").save("gs://dev-XXXX-analyt-XXXXXXXX") getting error :
Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token
what could be the reason or resolution. Need help
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-11-2025 03:30 PM
DId you find out why this was happening? I'm having the same issue
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-12-2025 12:34 AM
@KristiLogos - Use the cluster access mode as : Single User and not shared and try
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-12-2025 08:24 AM
@kiko_roy unfortunately that didnt' work. the error is stating its trying to get the access token from metadata server, I wonder why from the metadata server?
