Spark Bigquery Connector Availability
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-28-2024 02:30 AM
Hello,
We are trying to use this library (https://github.com/GoogleCloudDataproc/spark-bigquery-connector) to read Bigquery data from a Databricks cluster in Azure, could someone confirm if this library is fully available and supported on Databricks? From what we have reviewed, there is no problem in Google Cloud Dataproc but we would like to know if it is the same for Databricks.
In this link (https://docs.databricks.com/en/connect/external-systems/bigquery.html) it is indicated that the configuration is experimental, but it is not clear to me if it refers to the library itself or to the federated connections to Bigquery.
BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables. - GoogleCloudDataproc/spark-bigquery-connector