The json and command looks good to me.
Is it feasible for you to attach the full stacktrace, and/or check if this is applicable to your cluster setup as well?
https://community.databricks.com/t5/community-platform-discussions/bigquery-in-notebook-failing-with...
