cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Error writing data to Google Bigquery

udays22222
New Contributor II

Hi,

I am able to read data from a Bigquery table ,But am getting error writing data to a table in Bigquery

Followed instuctions in this document.

Connecting Databricks to BigQuery | Google Cloud

%scala

import scala.io.Source

val contentCred = "/dbfs/FileStore/keys/uday_123.json"

val jsonBytes = Source.fromFile(contentCred).getLines.mkString

 myEvents.write.format("bigquery").mode("overwrite").option("temporaryGcsBucket", "databricks-bq-789")

  .option("credentials", new String(java.util.Base64.getEncoder.encode(jsonBytes.getBytes)))

  .option("parentProject", "uday-***")

  .option("project", "uday-***") 

  .option("table","uday-***.uday_testdata.employee1")

 .save()

Error:

IOException: Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

Caused by: HttpResponseException: 404 Not Found

GET http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

ResourceNotFound

The specified resource does not exist.

'computeMetadata' isn't a valid resource name.

6 REPLIES 6

Kaniz
Community Manager
Community Manager

Hi @uday sโ€‹, There is a similar thread on our community with the best answer. Please have a look and let us know if that helps.

udays22222
New Contributor II

Thanks @Kaniz Fatmaโ€‹  for the reply.

I have followed the steps mentioned in the thread, But I got the same error

OException: Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

Caused by: HttpResponseException: 404 Not Found

Please note I am able to read data from Bigquery table, But not able to write data.

Could anyone reply on solution for this error

Cedric
Valued Contributor
Valued Contributor

Hi @uday sโ€‹,

I've encountered this issue earlier this week. Could you please ensure that your cluster configuration is correct? I was able to write to the BigQuery tables after following Step 2.

https://docs.databricks.com/data/data-sources/google/bigquery.html#step-2-set-up-databricks

Let me know if it works!

pierre
New Contributor II

I've the same issue. I can read a BigQuery table.

But when I'm trying to save data into bigquery : 

IOException: Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

Caused by: HttpResponseException: 404 Not Found

GET http://***.***.***.***/computeMetadata/v1/instance/service-accounts/default/token

http://www.w3.org/2001/XMLSchema-instance"; xmlns:xsd="http://www.w3.org/2001/XMLSchema">

  

ResourceNotFound

  The specified resource does not exist.

  'computeMetadata' isn't a valid resource name.

Anonymous
Not applicable

Hi @uday sโ€‹ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.