cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Online tables schema only contains string (varchar) columns

obitech01
New Contributor II

I created an online table for feature serving based on an existing delta table (used as the source table).

This source table contains a struct column and an array column, but when the online table is created, those two columns show up as strings columns (or varchar columns).

Is this the default behavior of an online table?

I don't understand why the online table doesn't match its source table from which it was created/synced.

Side note: (Image attached) I noticed that the data source type shown for the online table is MySql. I would have expected this to be a Delta source

1 ACCEPTED SOLUTION

Accepted Solutions

Alberto_Umana
Databricks Employee
Databricks Employee

Hello @obitech01,

The behavior you are observing with the struct and array columns being converted to string (or varchar) columns in the online table is indeed the default behavior. An online table is a read-only copy of a Delta Table that is stored in row-oriented format optimized for online access. Those complex data types like struct and array are not preserved in their original form but are instead converted to string types

Additionally, the data source type for online tables is MySQL, so it shows MySQL instead of Delta

https://docs.databricks.com/en/machine-learning/feature-store/online-tables.html

View solution in original post

2 REPLIES 2

Alberto_Umana
Databricks Employee
Databricks Employee

Hello @obitech01,

The behavior you are observing with the struct and array columns being converted to string (or varchar) columns in the online table is indeed the default behavior. An online table is a read-only copy of a Delta Table that is stored in row-oriented format optimized for online access. Those complex data types like struct and array are not preserved in their original form but are instead converted to string types

Additionally, the data source type for online tables is MySQL, so it shows MySQL instead of Delta

https://docs.databricks.com/en/machine-learning/feature-store/online-tables.html

Makes sense. Thanks for the clarification, this helps.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group