Looking to learn how you can use responsible AI toolkits on Databricks? Interested in learning how you can incorporate open source tools like SHAP and Fairlearn with Databricks?I would recommend checking out this blog: Mitigating Bias in Machine Lear...
I have saved a model in the model registry using MLFlow. How can I find the shap values for this model once I have generated predictions in batch mode? Shap tree explainer does not support the mlflow pyfunc model type. When I use mlflow.shap.log_exp...
Hi, Could you please check if this section helps in the below documentation:https://www.databricks.com/blog/2019/06/17/detecting-bias-with-shap.html
Hi! This is kind of an urgent question so any help would be greatly appreciated! Thanks so much! So I'm following this tutorial to try to create an MLflow project: https://docs.databricks.com/applications/mlflow/projects.htmlI tried with the example ...
This is generally not how you use MLflow in Databricks. You are already in Databricks so do not need to send code to Databricks to execute. Instead just run your code in a notebook; there is no need to package as an MLflow Project. Projects are prima...
Hi! I was just wondering what are the parameters For MLflow Project file? I'm following this tutorial to create my own MLflow Project: https://docs.databricks.com/applications/mlflow/projects.htmland within this tutorial, the MLproject file looks lik...
These parameters are parameters that you will specify when you run the MLflow Project with the mlflow CLI. It lets you parameterize your code, and then pass different parameters to it. How you use them is up to your code. These are not model hyperpar...
I have a feature store table and I would like to change one of the features from IntegerType to FloatType, I can't merge this change as it violates the schema. Is it possible to drop a single feature from the table and add the revised feature?Current...
Hi there @_ _​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
Versions of Code:Databricks: 7.3 LTS ML (includes Apache Spark 3.0.1, Scala 2.12)AWS EMR: 6.1.0 (Spark 3.0.0, Scala 2.12)https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-610-release.html The problem:Errors in Databricks when replicating job th...
Time Series AutoML currently has very few models for comparison. How can I add some custom models into the mix so that they are compared each time?
could you please let us know what are the custom models you are looking to add to time series AutoML?
The curated zone is pushed to cloud data warehouse such as Synapse Dedicated SQL Pools which then acts as a serving layer for BI tools and analyst.I believe we can have models in gold layer and have BI connect to this layer or we can have serverless ...
Thank you, so for a large workload, where we need lot of optimization we might need Synapse, but for a small/medium workload, we might have to stick to Delta Table
Hello,Please suggest how to obtain name of primary key columns in my feature store table in hive metastore.'describe' gives me the name of the columns but not the indexesThanks in advance for your help.Regards,
Hi Vidula,Yes, I solved the query !I used the function getTable of class FeatureStoreClient(). You have all datas you need : primary keys, timestamp_keys, features of feature store table.Regards
We are trying to write the data frame to s3 using: df.write.mode('overwrite').save("s3://BUCKET-NAME/temp"), but recently we are getting the following error: 'com.amazonaws.services.s3.model.MultiObjectDeleteException: One or more objects could not ...
Hi @Mayank Kasturia​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
Hi!If I need to use many workers to distributes regular pandas, I would use a pandas_UDF. (having regular python crunching a slice of my data, on each node, and combining all results back to the driver node)Is there something equivalent for R?Thanks,
Hi Team,I am trying to implement automl in python for my timeseries forecast problem.But, I was facing below error during the model training:AttributeError: 'StanModel' object has no attribute 'fit_class'Due to the above error, the experiment failed ...
Hi @Prakash Thavamurugan​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...
from databricks import feature_storeI am trying to import feature_store but it is showing this error.ImportError: cannot import name 'feature_store' from 'databricks' (/databricks/python/lib/python3.8/site-packages/databricks/__init__.py)
Is this issue resolved completely? We are facing the same problem. this might help.
Hello, I am very new with databricks and MLflow. I faced with the problem about running job. When the job is run, it usually failed and retried itself, so it incasesed running time, i.e., from normally 6 hrs to 12-18 hrs. From the error log, it shows...
Hey there @Tanawat Benchasirirot​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hea...
I'm in a Data Science Bootcamp, and the final case study includes data preprocessing (done), using a linear regression model on the data, then porting to SQL for visualization. The model build uses custom python code provided as part of the exercise....
Hi @Joe DiGiovanni​ Just wanted to check in if you were able to resolve your issue or do you need more help? We'd love to hear from you.Thanks!
