cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Feature Store : for sklearn flavored models, are timestamps fully supported?

thib
New Contributor III

I have created a feature table (Databricks runtime ML 10.2) that includes a timestamp column as a primary key, that is not used as a feature but as a column to join on.

I have then created a model that trains from this feature table and some additional data, which excludes the primary keys. I tried excluding them, both using the feature store api, and from the sklearn api. The model is being trained fine, but when use the score_batch() method, I get an error claiming that 'TypeError: float() argument must be a string or a number, not 'Timestamp''.

This error is coming from sklearn, so is there some incompatibility there, or is this a bug in feature store?

Steps to reproduce :

  • create feature table with one column as timestamp type
  • train a model using sklearn that does not use that timestamp column
  • use score_batch() method and visualize results
1 ACCEPTED SOLUTION

Accepted Solutions

Hubert-Dudek
Esteemed Contributor III

maybe you can just try to cast timestamp as int

View solution in original post

4 REPLIES 4

Hubert-Dudek
Esteemed Contributor III

maybe you can just try to cast timestamp as int

thib
New Contributor III

Thanks for your reply Hubert. Yes, casting it to long or int does solve the issue, but it is a workaround and I would like to keep the data as-is, with directly interpretable timestamps, especially when there is no reason why they should trigger an error during the prediction step since it is not being used at that stage.

jose_gonzalez
Databricks Employee
Databricks Employee

Hi @Thibault Daoulas​ ,

Did @Kaniz Fatma​ response help you to resolved your question? if yes, please mark it as best response. If not, please let us know.

thib
New Contributor III

Hi, it did not, but at least I know they are not fully supported so a workaround is to avoid timestamps, so I suppose you can mark this as resolved

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group