cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Online tables schema only contains string (varchar) columns

obitech01
New Contributor II

I created an online table for feature serving based on an existing delta table (used as the source table).

This source table contains a struct column and an array column, but when the online table is created, those two columns show up as strings columns (or varchar columns).

Is this the default behavior of an online table?

I don't understand why the online table doesn't match its source table from which it was created/synced.

Side note: (Image attached) I noticed that the data source type shown for the online table is MySql. I would have expected this to be a Delta source

1 ACCEPTED SOLUTION

Accepted Solutions

Alberto_Umana
Databricks Employee
Databricks Employee

Hello @obitech01,

The behavior you are observing with the struct and array columns being converted to string (or varchar) columns in the online table is indeed the default behavior. An online table is a read-only copy of a Delta Table that is stored in row-oriented format optimized for online access. Those complex data types like struct and array are not preserved in their original form but are instead converted to string types

Additionally, the data source type for online tables is MySQL, so it shows MySQL instead of Delta

https://docs.databricks.com/en/machine-learning/feature-store/online-tables.html

View solution in original post

2 REPLIES 2

Alberto_Umana
Databricks Employee
Databricks Employee

Hello @obitech01,

The behavior you are observing with the struct and array columns being converted to string (or varchar) columns in the online table is indeed the default behavior. An online table is a read-only copy of a Delta Table that is stored in row-oriented format optimized for online access. Those complex data types like struct and array are not preserved in their original form but are instead converted to string types

Additionally, the data source type for online tables is MySQL, so it shows MySQL instead of Delta

https://docs.databricks.com/en/machine-learning/feature-store/online-tables.html

Makes sense. Thanks for the clarification, this helps.

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now