cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Cirsa
by New Contributor II
  • 3526 Views
  • 3 replies
  • 2 kudos

Resolved! Problem creating FeatureStore

Hi,When trying to create the first table in the Feature Store i get a message: ''DataFrame' object has no attribute 'isEmpty'... but it is not. So I cannot use the function: feature_store.create_table()With this code you should be able to reproduce t...

  • 3526 Views
  • 3 replies
  • 2 kudos
Latest Reply
Cirsa
New Contributor II
  • 2 kudos

@Hubert Dudek​Sry about the 'df_train', I forgot to change it (the error I commented is real with the proper DF). Changing the DBR to 11.3 LTS solved the problem. Thanks!

  • 2 kudos
2 More Replies
Ashley1
by Contributor
  • 7368 Views
  • 5 replies
  • 4 kudos

Unity Catalog - existing dbfs mounts and feature store

Hi All, We're currently considering turning on Unity Catalog but before we flick the switch I'm hoping I can get a bit more confidence of what will happen with our existing dbfs mounts and feature store. The bit that makes me nervous is the crede...

  • 7368 Views
  • 5 replies
  • 4 kudos
Latest Reply
karthik_p
Esteemed Contributor
  • 4 kudos

@Ashley Betts​ can you please check below article, as far as i know we can use external mount points by configuring storage credentials in unity catalog . default method is managed tables, but we can point external tables also. 1. you can upgrade exi...

  • 4 kudos
4 More Replies
sameer_gupta
by New Contributor
  • 2043 Views
  • 3 replies
  • 0 kudos

Error in importing feature_store

from databricks import feature_storeI am trying to import feature_store but it is showing this error.ImportError: cannot import name 'feature_store' from 'databricks' (/databricks/python/lib/python3.8/site-packages/databricks/__init__.py)

  • 2043 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Is this issue resolved completely? We are facing the same problem. this might help.

  • 0 kudos
2 More Replies
mayank347
by New Contributor II
  • 1593 Views
  • 1 replies
  • 3 kudos

Feature Store - Feature Lookup with Filter

I am working with feature store to save the engineered features. However, for the specific case we have lots of feature table and lot of separate target variables on which we want to train separate models. Now for each of these model, we can leverage...

  • 1593 Views
  • 1 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Thanks for taking the time to let us know how to make Databricks even better! @Mayank Srivastava​  I love that you included a real-life example as well. I think I know the right PM at Databricks that will be interested in this input. Thanks again for...

  • 3 kudos
Slalom_Tobias
by New Contributor III
  • 2780 Views
  • 2 replies
  • 3 kudos

Resolved! ML Practioner | ML 10 - Feature Store notebook | feature_store import error

the following code...from pyspark.sql.functions import monotonically_increasing_id, lit, expr, randimport uuidfrom databricks import feature_storefrom pyspark.sql.types import StringType, DoubleTypefrom databricks.feature_store import feature_table, ...

  • 2780 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hope that was an easy fix - @Tobias Cortese​ ! Thanks for marking the "best answer"!

  • 3 kudos
1 More Replies
vives
by New Contributor II
  • 3464 Views
  • 3 replies
  • 1 kudos

How to add signature to model logged through feature store?

It seems that the current method log_model from the FeatureStoreClient class lacks a way to pass in the model signature (as opposed as doing it through mlflow directly). Is there a workaround to append this information? Thanks!

  • 3464 Views
  • 3 replies
  • 1 kudos
Latest Reply
AmirLyall
Databricks Employee
  • 1 kudos

Hello!You can log a model with a signature by passing a signature object as an argument with your log_model call. Please see here.Here's an example of this in action in a databricks notebook.Hope that helps!-Amir

  • 1 kudos
2 More Replies
Nithin
by New Contributor II
  • 12088 Views
  • 7 replies
  • 1 kudos

How to access databricks feature store outside databricks?

We are building the feature store using databricks API. Few of the machine learning engineers are using Jupyter notebooks. Is it possible to use feature store outside databricks?

  • 12088 Views
  • 7 replies
  • 1 kudos
Latest Reply
datariel
New Contributor II
  • 1 kudos

Hi @Kaniz Fatma​  and @Jose Gonzalez​ ,turning back to the original question, and considering that one of the main benefits of the Feature Store is the removal of the online/offline skew, how could I access to the features from a client application l...

  • 1 kudos
6 More Replies
Direo
by Contributor II
  • 1613 Views
  • 1 replies
  • 2 kudos

Feature store errors

HiWhen I open feature store, I get an error saying that "Failed to load some job schedules". When I open one of the feature store tables, I get several additional errors:"Failed to laod lates run for some job producers","Failed to laod some job produ...

image
  • 1613 Views
  • 1 replies
  • 2 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 2 kudos

@Direo Direo​ , In that case, I would write to support (in the case of Azure to Microsoft support).

  • 2 kudos
romanzdk
by New Contributor II
  • 2489 Views
  • 1 replies
  • 0 kudos

Databricks online store - Login to Azure SQL Database with Service Principal

I want to use Databricks Online Store with Azure SQL Database, however I am unable to autenthicate through Databricks Feature Store API. I need to use Service Principal credentials.I tried using Application ID as username and Secret as password, but ...

  • 2489 Views
  • 1 replies
  • 0 kudos
Latest Reply
romanzdk
New Contributor II
  • 0 kudos

no one?

  • 0 kudos
mhansinger
by New Contributor II
  • 19198 Views
  • 4 replies
  • 1 kudos

Resolved! Set default "spark.driver.maxResultSize" from the notebook

Hello,I would like to set the default "spark.driver.maxResultSize" from the notebook on my cluster. I know I can do that in the cluster settings, but is there a way to set it by code?I also know how to do it when I start a spark session, but in my ca...

  • 19198 Views
  • 4 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Maximilian Hansinger​ Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark the answer as best? If not, please tell us so we can help you.Thanks!

  • 1 kudos
3 More Replies
thib
by New Contributor III
  • 3429 Views
  • 4 replies
  • 2 kudos

Resolved! Feature Store : for sklearn flavored models, are timestamps fully supported?

I have created a feature table (Databricks runtime ML 10.2) that includes a timestamp column as a primary key, that is not used as a feature but as a column to join on.I have then created a model that trains from this feature table and some additiona...

  • 3429 Views
  • 4 replies
  • 2 kudos
Latest Reply
thib
New Contributor III
  • 2 kudos

Hi, it did not, but at least I know they are not fully supported so a workaround is to avoid timestamps, so I suppose you can mark this as resolved

  • 2 kudos
3 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 3552 Views
  • 1 replies
  • 32 kudos

Databricks Roadmap Azure There are a lot of excitement new features coming in 2022. I tried to put them all on one list: Unity catalog (seems that it ...

Databricks Roadmap AzureThere are a lot of excitement new features coming in 2022. I tried to put them all on one list:Unity catalog (seems that it will exists next to hive metastore and it will be possible to migrate)Control metastore, unity creatio...

  • 3552 Views
  • 1 replies
  • 32 kudos
Jack_Watson
by Contributor
  • 9745 Views
  • 4 replies
  • 0 kudos

Resolved! I am saving a new feature table to the Databricks feature store, and it won't write the data sources of the tables used to create the feature table, because they are Hive tables that point to Azure Data Lake Storage Gen1 Delta tables

My notebook is pulling in Hive tables from DBFS, that point to ADLS Gen1 file locations for their data (Delta tables), creating the feature table as a data frame within the notebook, then calling on the feature store client to save down the feature t...

  • 9745 Views
  • 4 replies
  • 0 kudos
Latest Reply
Atanu
Databricks Employee
  • 0 kudos

@Jack Watson​  Could you please confirm the write is succeeding ? If yes, as per my understanding This is a warning for some validation that we will be removing shortly. We’ll likely remove the validation which save the data source.Thanks.

  • 0 kudos
3 More Replies
thib
by New Contributor III
  • 2035 Views
  • 3 replies
  • 4 kudos

Resolved! Feature store : Can create_training_set() be implemented to execute an inner join?

For timeseries feature tables, an inner join is made at the creation of the feature table. For the other type of feature tables, a left join is made, so NaN values can show up in the training set. Can the inner join in create_training_set() method be...

  • 2035 Views
  • 3 replies
  • 4 kudos
Latest Reply
thib
New Contributor III
  • 4 kudos

Thank you Hubert, that's a good alternative, I just thought I'd stick to the api as much as possible, but this solves it.

  • 4 kudos
2 More Replies
Edmondo
by New Contributor III
  • 1919 Views
  • 0 replies
  • 0 kudos

MlFlow and Feature Store: mlflow.spark.autolog, using feature store on Databricks, FeatureStoreClient.log_model()?

As I am moving my first steps within the Databricks Machine Learning Workspace, I am getting confused by some features that by "documentation" seem to overlap. Does autolog for spark on mlflow provide different tracking than using a training set crea...

  • 1919 Views
  • 0 replies
  • 0 kudos
Labels