Job compute fails due to BQ permissions

ruia-dojo
New Contributor

Hello,

My databricks workspace is associated to GCP project analytics.

But me and my team mostly work on GCP project data-science, which contains the only BQ dataset that we have write access to.

I'm trying to automate a pipeline to run on job compute and it fails when reading a table from project data-science, since reading implies writing to a temporary table that we have set to be in the project and dataset we have the rights to (materialisation_project=data-science, dataset=dst). When running the notebook myself or using our development cluster, it works as intended. When running the notebook on a pipeline using job compute, it fails with the error 

Access Denied: Project analytics: User does not have bigquery.jobs.create permission in project analytics

What could be the issue here? job compute service account? How do I edit it? It's trying to write to GCP project analytics, which it shouldn't, given the materialisation parameters passed as an argument.

Thanks,

Rui