AnalysisException: Multiple sources found for bigquery (com.google.cloud.spark.bigquery.BigQueryRelationProvider, com.google.cloud.spark.bigquery.v2.BigQueryTableProvider), please specify the fully qualified class name.

[parulpaul]

While reading data from BigQuery to Databricks

getting the error : AnalysisException:

Multiple sources found for bigquery (com.google.cloud.spark.bigquery.BigQueryRelationProvider, com.google.cloud.spark.bigquery.v2.BigQueryTableProvider),

please specify the fully qualified class name.

Jar used - spark-bigquery-with-dependencies_2.12-0.27.0.jar

df = spark.read.format("bigquery")\

        .option("credentialsFile",credientialFilePath)\

        .option("ParentProject",projectName)\

        .option("table",tableName)\

        .load()

[Debayan]

Hi @Parul Paul​ , could you please check if this is the scenario: https://stackoverflow.com/questions/68623803/load-to-bigquery-via-spark-job-fails-with-an-exception-...

Also, you can refer: https://github.com/GoogleCloudDataproc/spark-bigquery-connector/issues/55

using big query connector with spark , please refer: https://cloud.google.com/dataproc/docs/tutorials/bigquery-connector-spark-example

Also, the issue can be: multiple versions of spark sql in the class path, which could be because of libraries installed on the cluster. Could you please check uninstalling few of the unnecessary libraries and restarting the cluster?