topic Is it possible to create a feature store training set directly from a feature store table? in Data Engineering

Is it possible to create a feature store training set directly from a feature store table?

lewit — Wed, 11 Jan 2023 08:52:01 GMT

Rather than joining features from different tables, I just wanted to use a single feature store table and select some of its features, but still log the model in the feature store. The problem I am facing is that I do not know how to create the training set without first creating another dataframe to join with features from the feature store.

Re: Is it possible to create a feature store training set directly from a feature store table?

lewit — Fri, 13 Jan 2023 07:46:59 GMT

I did check this but it does not help me sadly. My problem is not necessarily with the Feature Lookups part, but with the create_training_set() function. It demands a df with which one can join the features described by FeatureLookup().

In the example:

training_set = fs.create_training_set(

df=training_df,

feature_lookups = feature_lookups,

label = 'rating',

exclude_columns = ['customer_id', 'product_id'])

My problem is specifically with df. I have a feature store table that contains all the information I need, features and label, but as of now I need to create a separate dataframe with an index and my label of choice (which I am extracting from the feature store with read_table()), and then create a training_set joining it with the rest of the features I need from that table. This seems like an unnecessary step.

Isn't there a way to create the training set without having to create this "index" df first? If I use the multiple feature lookup trick I will still need to inform some df to the create_training_set() command for it to work since it is a mandatory parameter.

Re: Is it possible to create a feature store training set directly from a feature store table?

Debayan — Thu, 12 Jan 2023 21:03:09 GMT

Hi, Could you please refer https://docs.databricks.com/machine-learning/feature-store/train-models-with-feature-store.html#create-a-trainingset-using-the-same-feature-multiple-times and let us know if this helps.