cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Is it possible to create a feature store training set directly from a feature store table?

lewit
New Contributor II

Rather than joining features from different tables, I just wanted to use a single feature store table and select some of its features, but still log the model in the feature store. The problem I am facing is that I do not know how to create the training set without first creating another dataframe to join with features from the feature store.

2 REPLIES 2

Debayan
Esteemed Contributor III
Esteemed Contributor III

lewit
New Contributor II

I did check this but it does not help me sadly. My problem is not necessarily with the Feature Lookups part, but with the create_training_set() function. It demands a df with which one can join the features described by FeatureLookup().

In the example:

training_set = fs.create_training_set(

df=training_df,

feature_lookups = feature_lookups,

label = 'rating',

exclude_columns = ['customer_id', 'product_id'])

My problem is specifically with df. I have a feature store table that contains all the information I need, features and label, but as of now I need to create a separate dataframe with an index and my label of choice (which I am extracting from the feature store with read_table()), and then create a training_set joining it with the rest of the features I need from that table. This seems like an unnecessary step.

Isn't there a way to create the training set without having to create this "index" df first? If I use the multiple feature lookup trick I will still need to inform some df to the create_training_set() command for it to work since it is a mandatory parameter.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.