cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Is it possible to create a feature store training set directly from a feature store table?

lewit
New Contributor II

Rather than joining features from different tables, I just wanted to use a single feature store table and select some of its features, but still log the model in the feature store. The problem I am facing is that I do not know how to create the training set without first creating another dataframe to join with features from the feature store.

2 REPLIES 2

Debayan
Esteemed Contributor III
Esteemed Contributor III

lewit
New Contributor II

I did check this but it does not help me sadly. My problem is not necessarily with the Feature Lookups part, but with the create_training_set() function. It demands a df with which one can join the features described by FeatureLookup().

In the example:

training_set = fs.create_training_set(

df=training_df,

feature_lookups = feature_lookups,

label = 'rating',

exclude_columns = ['customer_id', 'product_id'])

My problem is specifically with df. I have a feature store table that contains all the information I need, features and label, but as of now I need to create a separate dataframe with an index and my label of choice (which I am extracting from the feature store with read_table()), and then create a training_set joining it with the rest of the features I need from that table. This seems like an unnecessary step.

Isn't there a way to create the training set without having to create this "index" df first? If I use the multiple feature lookup trick I will still need to inform some df to the create_training_set() command for it to work since it is a mandatory parameter.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!