cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Online Table: create only if it does not exist

lauraxyz
Contributor

i'm following this Doc to create online table using Databricks SDK. How can i set it to create ONLY when it doesn't exist, to avoid the failure of "table already exists" error? 

Or, is there another way to programatic way to check existence of an Online Table?

2 REPLIES 2

Alberto_Umana
Databricks Employee
Databricks Employee

Hello @lauraxyz,

There isn't a direct way to check if an online table already exists, but you can try with a try-except block to handle it:

For example:

from databricks.sdk import WorkspaceClient
from databricks.sdk.service.catalog import OnlineTable, OnlineTableSpec
from databricks.sdk.service import catalog

w = WorkspaceClient()

try:
# Attempt to get the online table
existing_table = w.online_tables.get('main.default.my_online_table')
print("Online table already exists.")
except catalog.NotFound:
# If the table doesn't exist, create it
spec = OnlineTableSpec(
primary_key_columns=["pk_col"],
source_table_full_name="main.default.source_table",
run_triggered=OnlineTableSpec.TriggeredSchedulingPolicy()
)

online_table = OnlineTable(
name="main.default.my_online_table",
spec=spec
)

w.online_tables.create_and_wait(table=online_table)
print("Online table created successfully.")

lauraxyz
Contributor

Thank you @Alberto_Umana ,  that's a good way to go when there's no built-in creat-if-not-exist feature.

i also tried a different way to use  information_schema, i think it should work too

def table_exists(table_name):
    return spark.sql(f"""
        select count(*) as count
        from system.information_schema.tables
        where table_catalog = '{catalog}'
        and table_schema = '{schema}'
        and table_name = '{online_table_name}'
    """).collect()[0]['count'] != 0

 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group