topic Online tables schema only contains string (varchar) columns in Machine Learning

Online tables schema only contains string (varchar) columns

obitech01 — Sun, 26 Jan 2025 22:19:54 GMT

I created an online table for feature serving based on an existing delta table (used as the source table).

This source table contains a struct column and an array column, but when the online table is created, those two columns show up as strings columns (or varchar columns).

Is this the default behavior of an online table?

I don't understand why the online table doesn't match its source table from which it was created/synced.

Side note: (Image attached) I noticed that the data source type shown for the online table is MySql. I would have expected this to be a Delta source

Re: Online tables schema only contains string (varchar) columns

Alberto_Umana — Mon, 27 Jan 2025 00:30:31 GMT

Hello @obitech01,

The behavior you are observing with the struct and array columns being converted to string (or varchar) columns in the online table is indeed the default behavior. An online table is a read-only copy of a Delta Table that is stored in row-oriented format optimized for online access. Those complex data types like struct and array are not preserved in their original form but are instead converted to string types

Additionally, the data source type for online tables is MySQL, so it shows MySQL instead of Delta

https://docs.databricks.com/en/machine-learning/feature-store/online-tables.html

Re: Online tables schema only contains string (varchar) columns

obitech01 — Mon, 27 Jan 2025 01:06:01 GMT

Makes sense. Thanks for the clarification, this helps.