cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Error writing data to Google Bigquery

udays22222
New Contributor II

Hi,

I am able to read data from a Bigquery table ,But am getting error writing data to a table in Bigquery

Followed instuctions in this document.

Connecting Databricks to BigQuery | Google Cloud

%scala

import scala.io.Source

val contentCred = "/dbfs/FileStore/keys/uday_123.json"

val jsonBytes = Source.fromFile(contentCred).getLines.mkString

 myEvents.write.format("bigquery").mode("overwrite").option("temporaryGcsBucket", "databricks-bq-789")

  .option("credentials", new String(java.util.Base64.getEncoder.encode(jsonBytes.getBytes)))

  .option("parentProject", "uday-***")

  .option("project", "uday-***") 

  .option("table","uday-***.uday_testdata.employee1")

 .save()

Error:

IOException: Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

Caused by: HttpResponseException: 404 Not Found

GET http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

ResourceNotFound

The specified resource does not exist.

'computeMetadata' isn't a valid resource name.

6 REPLIES 6

Kaniz_Fatma
Community Manager
Community Manager

Hi @uday s​, There is a similar thread on our community with the best answer. Please have a look and let us know if that helps.

Thanks @Kaniz Fatma​  for the reply.

I have followed the steps mentioned in the thread, But I got the same error

OException: Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

Caused by: HttpResponseException: 404 Not Found

Please note I am able to read data from Bigquery table, But not able to write data.

Could anyone reply on solution for this error

Cedric
Valued Contributor
Valued Contributor

Hi @uday s​,

I've encountered this issue earlier this week. Could you please ensure that your cluster configuration is correct? I was able to write to the BigQuery tables after following Step 2.

https://docs.databricks.com/data/data-sources/google/bigquery.html#step-2-set-up-databricks

Let me know if it works!

pierre
New Contributor II

I've the same issue. I can read a BigQuery table.

But when I'm trying to save data into bigquery : 

IOException: Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

Caused by: HttpResponseException: 404 Not Found

GET http://***.***.***.***/computeMetadata/v1/instance/service-accounts/default/token

http://www.w3.org/2001/XMLSchema-instance"; xmlns:xsd="http://www.w3.org/2001/XMLSchema">

  

ResourceNotFound

  The specified resource does not exist.

  'computeMetadata' isn't a valid resource name.

Anonymous
Not applicable

Hi @uday s​ 

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. 

We'd love to hear from you.

Thanks!

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!