@kfiryโ adding to @Werner Stinckensโ did you added projectid in read spark query , projectid should be one where big query instance running. also please follow best practices in terms of egress data cost
spark.read.format("bigquery") \
.option("table", table) \
.option("project", <project-id>) \
.option("parentProject", <parent-project-id>) \
.load()