cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

I've data in s3/Iceberg tables. How to read it using databricks SparkSQL ?

Nalapriya
New Contributor II

I tried this method: 

df = spark.read.format("iceberg").load("s3-bucket-path")
But got an error: Multiple sources found for iceberg (com.databricks.sql.transaction.tahoe.uniform.sources.IcebergBrowseOnlyDataSource, org.apache.iceberg.spark.source.IcebergSource), please specify the fully qualified class name.
Then updated the code like this: 
df = spark.read.format("org.apache.iceberg.spark.source.IcebergSource").load("s3-bucket-path")
Error:  The table or view `default_iceberg`.`s3://my-bucket-path`.`` cannot be found. Verify the spelling and correctness of the schema and catalog. If you did not qualify the name with a schema, verify the current_schema() output, or qualify the name with the correct schema and
How to resolve this issue and read data using databricks SparkSQL?
 
Note: I've installed the spark runtime jar library, configured the cluster with spark requirements and catalog setup as well.
3 REPLIES 3

Alberto_Umana
Databricks Employee
Databricks Employee

Hi @Nalapriya,

What is the DBR version? and library jar version?

Could you try with below code test?

# Set the catalog and schema
spark.sql("USE CATALOG my_catalog")
spark.sql("USE SCHEMA my_schema")

# Read the Iceberg table
df = spark.read.format("org.apache.iceberg.spark.source.IcebergSource").load("s3://my-bucket-path")

Hi,

When using Unity catalog, I'm unable to create a new catalog that I can use for querying an Iceberg table.

How can I resolve this issue?

The Iceberg table is stored in S3.

Nalapriya
New Contributor II

Hi @Alberto_Umana

I tried the steps you've provided but still I'm not able to read data which is in iceberg format. It would be useful if I get any other suggestions.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group