Online Table Migration

[nulltype]

I am currently trying to migrate our Online Tables to synced tables with Online Feature Store since Online Tables is deprecated. 

When creating a new table, it worked just fine and how the docs said it would (https://docs.databricks.com/aws/en/machine-learning/feature-store/online-feature-store)

But now I am trying to create a pipeline that will migrate our tables by creating back up Online Tables and feature specs while also referencing these docs (https://docs.databricks.com/aws/en/machine-learning/feature-store/migrate-from-online-tables). But when publishing the table to our Online Store, the online table doesnt populate in UC. Which causes errors when trying to update our endpoint

The code below will return a success message of 

PublishedTable: online_table_name='online_table', pipeline_id=None

online_store = fe.get_online_store(name="online_store")

fe.publish_table(

online_store=online_store,

source_table_name="offline_table",

online_table_name="online_table",

)

The code below will return an Error of `NotFound: Table 'online_table does not exist.`

w.online_tables.get("online_table")

Not sure if I am doing something wrong with publishing the table or not but some insight or points in the right direction would be great!

[mark_ott · Accepted solution]

Migrating from deprecated Online Tables to synced tables with the Databricks Online Feature Store can be tricky due to several points of integration and timing between Unity Catalog (UC), Feature Store metadata, and the underlying online store. The main problems described here are:

1. The publish_table call succeeds, but the table doesn't appear in UC.

2. Retrieving the table via w.online_tables.get("online_table") throws a NotFound error.

Here are some targeted insights and troubleshooting suggestions:

Understanding Online Table Migration

The migration process from Online Tables involves backing up existing tables and feature specs, then publishing those tables to the new Online Feature Store-backed storage solution. Unity Catalog is the interface for table discovery and access, so if the table isn’t visible there, downstream queries and endpoint updates will fail.

Common Issues with publish_table

• Publish Success, Missing Table: If publish_table returns success but UC does not show the table, it means the metadata update was not fully propagated OR there’s a misalignment between Feature Store and UC registration.

• Online Store Object: The result may be “published” in the managed online store, but not fully registered with Unity Catalog, leaving endpoint integrations broken.

Troubleshooting Steps

• Check for Propagation Delay: Sometimes, it takes several minutes for updates to fully propagate and appear in Unity Catalog. Try waiting 5–10 minutes and refreshing the UC listing.

• Check the Table Location: Ensure online_table is being published to the correct schema and catalog in UC. Sometimes, default settings cause the table to be registered elsewhere. Use the fully qualified name, e.g., "catalog.schema.online_table".

• Validate Table Existence in UC: Before calling w.online_tables.get("online_table"), check in UC directly through the Databricks UI or SQL statement:

  text

  SHOW TABLES IN <catalog>.<schema>;

• Compare API vs. UI: Sometimes the Feature Store Python API is slightly out of sync with Unity Catalog state. The docs recommend always checking both until migration is complete.

Sample Migration Pipeline Steps

1. Back Up the Table & Features Spec
   Export the full feature table definition and store it offline.

2. Create Synced Table in Feature Store
   Use Feature Store APIs to create synced table object in UC.

3. Publish to Online Store
   Ensure the online_table_name and its fully qualified UC path are used and match the new storage location’s UC registration.
   Example:

   python

   fe.publish_table( online_store=online_store, source_table_name="offline_table", online_table_name="catalog.schema.online_table", )

4. Validate UC Listing
   List tables in UC and search for the correct name and schema.

5. Update Endpoint
   Use the newly published, fully-qualified table name for endpoint updates.

Key Best Practices

• Always use fully qualified table names in the form <catalog>.<schema>.<table> during migration and endpoint configuration.

• Monitor for propagation delays between Feature Store and Unity Catalog.

• Ensure that required permissions exist for both the pipeline service and the target UC schema.

Official Documentation References

The migration guide makes clear that online table publishing should synchronize with UC registration, but explicit validation via catalog and schema is often needed for endpoint updates. Directly referencing both migration and online feature store docs will help avoid silent sync errors.

---

In summary, make sure to use fully qualified table names at every stage, validate directly in UC, and be aware of propagation delays. If the issue persists after these checks, consider opening a support ticket with Databricks, as further troubleshooting may require insight into backend synchronization.