I've data in s3/Iceberg tables. How to read it using databricks SparkSQL ?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-11-2025 04:22 AM
I tried this method:
df = spark.read.format("iceberg").load("s3-bucket-path")
But got an error: Multiple sources found for iceberg (com.databricks.sql.transaction.tahoe.uniform.sources.IcebergBrowseOnlyDataSource, org.apache.iceberg.spark.source.IcebergSource), please specify the fully qualified class name.
Then updated the code like this:
df = spark.read.format("org.apache.iceberg.spark.source.IcebergSource").load("s3-bucket-path")
Error: The table or view `default_iceberg`.`s3://my-bucket-path`.`` cannot be found. Verify the spelling and correctness of the schema and catalog. If you did not qualify the name with a schema, verify the current_schema() output, or qualify the name with the correct schema and
How to resolve this issue and read data using databricks SparkSQL?
Note: I've installed the spark runtime jar library, configured the cluster with spark requirements and catalog setup as well.
