iceberg

Mohsen — Tue, 12 Sep 2023 01:18:19 GMT

Hi fellas
i am working on databricks using iceberg
at first i have configured my notebook as below
spark.conf.set("spark.sql.catalog.spark_catalog","org.apache.iceberg.spark.SparkCatalog")
spark.conf.set("spark.sql.catalog.spark_catalog.type", "hadoop")
spark.conf.set("spark.sql.catalog.spark_catalog.warehouse", "dbfs:/TPCDSIceberg/Data/")
then i created an iceberg tables

spark.sql(s"CREATE DATABASE IF NOT EXISTS bronzedb") spark.sql(s"DROP TABLE IF EXISTS bronzedb.bronzeTable") // Create an empty Delta table in the bronze layer spark.sql( s""" |CREATE TABLE bronzedb.bronzeTable($schemas) |USING $sinkFormat |location 'dbfs:/TPCDSIceberg/Data/bronzedb/bronzeTableName' |""".stripMargin )

and then i tried to load this table

val icebergDf = spark.read.format("iceberg").load(s"bronzedb.bronze_call_center")

but i get this Exception

RuntimeMetaException: Failed to connect to Hive Metastore Caused by: RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient Caused by: InvocationTargetException: Caused by: MetaException: Version information not found in metastore.

topic iceberg in Get Started Discussions

iceberg