topic Re: Error writing data to Google Bigquery in Data Engineering

Error writing data to Google Bigquery

udays22222 — Fri, 30 Sep 2022 19:25:03 GMT

Hi,

I am able to read data from a Bigquery table ,But am getting error writing data to a table in Bigquery

Followed instuctions in this document.

Connecting Databricks to BigQuery | Google Cloud

%scala

import scala.io.Source

val contentCred = "/dbfs/FileStore/keys/uday_123.json"

val jsonBytes = Source.fromFile(contentCred).getLines.mkString

myEvents.write.format("bigquery").mode("overwrite").option("temporaryGcsBucket", "databricks-bq-789")

.option("credentials", new String(java.util.Base64.getEncoder.encode(jsonBytes.getBytes)))

.option("parentProject", "uday-***")

.option("project", "uday-***")

.option("table","uday-***.uday_testdata.employee1")

.save()

Error:

IOException: Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

Caused by: HttpResponseException: 404 Not Found

GET http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

ResourceNotFound

The specified resource does not exist.

'computeMetadata' isn't a valid resource name.

Re: Error writing data to Google Bigquery

udays22222 — Mon, 03 Oct 2022 06:13:16 GMT

Thanks @Kaniz Fatma for the reply.

I have followed the steps mentioned in the thread, But I got the same error

OException: Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

Caused by: HttpResponseException: 404 Not Found

Please note I am able to read data from Bigquery table, But not able to write data.

Re: Error writing data to Google Bigquery

udays22222 — Tue, 04 Oct 2022 06:12:24 GMT

Could anyone reply on solution for this error

Re: Error writing data to Google Bigquery

Cedric — Fri, 07 Oct 2022 15:40:31 GMT

Hi @uday s,

I've encountered this issue earlier this week. Could you please ensure that your cluster configuration is correct? I was able to write to the BigQuery tables after following Step 2.

https://docs.databricks.com/data/data-sources/google/bigquery.html#step-2-set-up-databricks

Let me know if it works!

Re: Error writing data to Google Bigquery

Anonymous — Sun, 23 Oct 2022 06:05:59 GMT

Hi @uday s

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: Error writing data to Google Bigquery

pierre — Fri, 10 Feb 2023 18:38:35 GMT

I've the same issue. I can read a BigQuery table.

But when I'm trying to save data into bigquery :

IOException: Error getting access token from metadata server at: http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token

Caused by: HttpResponseException: 404 Not Found

GET http://***.***.***.***/computeMetadata/v1/instance/service-accounts/default/token

http://www.w3.org/2001/XMLSchema-instance"; xmlns:xsd="http://www.w3.org/2001/XMLSchema">

ResourceNotFound

The specified resource does not exist.

'computeMetadata' isn't a valid resource name.

Re: Error writing data to Google Bigquery

GeoPer — Fri, 13 Dec 2024 17:27:36 GMT

@udays22222 did you find any solution on this one? I face the same problem when I use Shared (Access mode) cluster. I can read but I cannot write with the error you mentioned.