cancel
Showing results forĀ 
Search instead forĀ 
Did you mean:Ā 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forĀ 
Search instead forĀ 
Did you mean:Ā 

Error writing data to Google Bigquery

udays22222
New Contributor II

Hi,

I am able to read data from a Bigquery table ,But am getting error writing data to a table in Bigquery

Followed instuctions in this document.

Connecting Databricks to BigQuery | Google Cloud

%scala

import scala.io.Source

val contentCred = "/dbfs/FileStore/keys/uday_123.json"

val jsonBytes = Source.fromFile(contentCred).getLines.mkString

 myEvents.write.format("bigquery").mode("overwrite").option("temporaryGcsBucket", "databricks-bq-789")

  .option("credentials", new String(java.util.Base64.getEncoder.encode(jsonBytes.getBytes)))

  .option("parentProject", "uday-***")

  .option("project", "uday-***") 

  .option("table","uday-***.uday_testdata.employee1")

 .save()

Error:

IOException: Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

Caused by: HttpResponseException: 404 Not Found

GET http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

ResourceNotFound

The specified resource does not exist.

'computeMetadata' isn't a valid resource name.

5 REPLIES 5

Thanks @Kaniz Fatmaā€‹  for the reply.

I have followed the steps mentioned in the thread, But I got the same error

OException: Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

Caused by: HttpResponseException: 404 Not Found

Please note I am able to read data from Bigquery table, But not able to write data.

Could anyone reply on solution for this error

Cedric
Databricks Employee
Databricks Employee

Hi @uday sā€‹,

I've encountered this issue earlier this week. Could you please ensure that your cluster configuration is correct? I was able to write to the BigQuery tables after following Step 2.

https://docs.databricks.com/data/data-sources/google/bigquery.html#step-2-set-up-databricks

Let me know if it works!

pierre
New Contributor II

I've the same issue. I can read a BigQuery table.

But when I'm trying to save data into bigquery : 

IOException: Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

Caused by: HttpResponseException: 404 Not Found

GET http://***.***.***.***/computeMetadata/v1/instance/service-accounts/default/token

http://www.w3.org/2001/XMLSchema-instance"; xmlns:xsd="http://www.w3.org/2001/XMLSchema">

  

ResourceNotFound

  The specified resource does not exist.

  'computeMetadata' isn't a valid resource name.

Anonymous
Not applicable

Hi @uday sā€‹ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonā€™t want to miss the chance to attend and share knowledge.

If there isnā€™t a group near you, start one and help create a community that brings people together.

Request a New Group