Hi @soumiknow 

Thank you for sharing the detailed context! To better understand and address the issue, let’s start by clarifying a few points. Are you able to successfully query BigQuery using the default spark-bigquery-connector library, even with the 0.22.2-SNAPSHOT version? This will help confirm whether the connectivity (network/authorization) itself is working or if there’s a broader issue with the Databricks setup. If there are any network restrictions, you'll need to ensure all traffic to Google Cloud endpoints is allowed in your networking setup.

Let me know:

• If the default library works as expected.
• Whether the issue is related to authentication or network configurations.

You may run these troubleshooting steps from a notebook:

https://cloud.google.com/compute/docs/troubleshooting/troubleshoot-metadata-server#rest