AnalysisException: Multiple sources found for bigquery (com.google.cloud.spark.bigquery.BigQueryRelationProvider, com.google.cloud.spark.bigquery.v2.BigQueryTableProvider), please specify the fully qualified class name.

[parulpaul]

While reading data from BigQuery to Databricks

getting the error : AnalysisException:

Multiple sources found for bigquery (com.google.cloud.spark.bigquery.BigQueryRelationProvider, com.google.cloud.spark.bigquery.v2.BigQueryTableProvider),

please specify the fully qualified class name.

Jar used - spark-bigquery-with-dependencies_2.12-0.27.0.jar

df = spark.read.format("bigquery")\

        .option("credentialsFile",credientialFilePath)\

        .option("ParentProject",projectName)\

        .option("table",tableName)\

        .load()

[Debayan]

Hi @Parul Paul​ , could you please check if this is the scenario: https://stackoverflow.com/questions/68623803/load-to-bigquery-via-spark-job-fails-with-an-exception-...

Also, you can refer: https://github.com/GoogleCloudDataproc/spark-bigquery-connector/issues/55

using big query connector with spark , please refer: https://cloud.google.com/dataproc/docs/tutorials/bigquery-connector-spark-example

Also, the issue can be: multiple versions of spark sql in the class path, which could be because of libraries installed on the cluster. Could you please check uninstalling few of the unnecessary libraries and restarting the cluster?

[Kaniz]

Hi @Parul Paul​ ​, We haven’t heard from you since the last response from @Debayan Mukherjee​ , and I was checking back to see if you have a resolution yet.

If you have any solution, please share it with the community as it can be helpful to others. Otherwise, we will respond with more details and try to help.

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.