cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

confusedIntern
by New Contributor III
  • 2651 Views
  • 4 replies
  • 0 kudos

What are the parameters For MLflow Project file

Hi! I was just wondering what are the parameters For MLflow Project file? I'm following this tutorial to create my own MLflow Project: https://docs.databricks.com/applications/mlflow/projects.htmland within this tutorial, the MLproject file looks lik...

  • 2651 Views
  • 4 replies
  • 0 kudos
Latest Reply
sean_owen
Databricks Employee
  • 0 kudos

These parameters are parameters that you will specify when you run the MLflow Project with the mlflow CLI. It lets you parameterize your code, and then pass different parameters to it. How you use them is up to your code. These are not model hyperpar...

  • 0 kudos
3 More Replies
deep_thought
by Contributor
  • 2246 Views
  • 3 replies
  • 3 kudos

Resolved! How to drop single feature from feature store table

I have a feature store table and I would like to change one of the features from IntegerType to FloatType, I can't merge this change as it violates the schema. Is it possible to drop a single feature from the table and add the revised feature?Current...

  • 2246 Views
  • 3 replies
  • 3 kudos
Latest Reply
Vidula
Honored Contributor
  • 3 kudos

Hi there @_ _​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 3 kudos
2 More Replies
643926
by New Contributor II
  • 1117 Views
  • 0 replies
  • 1 kudos

Substantial performance issues/degradation on Databricks when migrating job over to EMR

Versions of Code:Databricks: 7.3 LTS ML (includes Apache Spark 3.0.1, Scala 2.12)AWS EMR: 6.1.0 (Spark 3.0.0, Scala 2.12)https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-610-release.html The problem:Errors in Databricks when replicating job th...

  • 1117 Views
  • 0 replies
  • 1 kudos
ssk121995
by New Contributor
  • 937 Views
  • 1 replies
  • 0 kudos

How can I add custom models to Time Series AutoML?

Time Series AutoML currently has very few models for comparison. How can I add some custom models into the mix so that they are compared each time?

  • 937 Views
  • 1 replies
  • 0 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 0 kudos

could you please let us know what are the custom models you are looking to add to time series AutoML?

  • 0 kudos
Shuvi
by New Contributor III
  • 2720 Views
  • 3 replies
  • 5 kudos

Resolved! What is the use case of having Azure Synapse(DWH) and Delta Lake ( Gold) given we can connect BI to delta directly

The curated zone is pushed to cloud data warehouse such as Synapse Dedicated SQL Pools which then acts as a serving layer for BI tools and analyst.I believe we can have models in gold layer and have BI connect to this layer or we can have serverless ...

  • 2720 Views
  • 3 replies
  • 5 kudos
Latest Reply
Shuvi
New Contributor III
  • 5 kudos

Thank you, so for a large workload, where we need lot of optimization we might need Synapse, but for a small/medium workload, we might have to stick to Delta Table

  • 5 kudos
2 More Replies
NSRBX
by Contributor
  • 4375 Views
  • 3 replies
  • 2 kudos

Resolved! How to extract name of Primary Key Columns of feature store table in Databricks environment ?

Hello,Please suggest how to obtain name of primary key columns in my feature store table in hive metastore.'describe' gives me the name of the columns but not the indexesThanks in advance for your help.Regards,

  • 4375 Views
  • 3 replies
  • 2 kudos
Latest Reply
NSRBX
Contributor
  • 2 kudos

Hi Vidula,Yes, I solved the query !I used the function getTable of class FeatureStoreClient(). You have all datas you need : primary keys, timestamp_keys, features of feature store table.Regards

  • 2 kudos
2 More Replies
565050
by New Contributor
  • 3584 Views
  • 2 replies
  • 0 kudos

Can't overwrite to S3 object

We are trying to write the data frame to s3 using: df.write.mode('overwrite').save("s3://BUCKET-NAME/temp"), but recently we are getting the following error: 'com.amazonaws.services.s3.model.MultiObjectDeleteException: One or more objects could not ...

  • 3584 Views
  • 2 replies
  • 0 kudos
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Mayank Kasturia​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 0 kudos
1 More Replies
yopbibo
by Contributor II
  • 1325 Views
  • 1 replies
  • 0 kudos

Sending R functions to worker nodes

Hi!If I need to use many workers to distributes regular pandas, I would use a pandas_UDF. (having regular python crunching a slice of my data, on each node, and combining all results back to the driver node)Is there something equivalent for R?Thanks,

  • 1325 Views
  • 1 replies
  • 0 kudos
898495
by New Contributor
  • 1664 Views
  • 2 replies
  • 1 kudos

MLFlow Python Error for Forecast problem

Hi Team,I am trying to implement automl in python for my timeseries forecast problem.But, I was facing below error during the model training:AttributeError: 'StanModel' object has no attribute 'fit_class'Due to the above error, the experiment failed ...

  • 1664 Views
  • 2 replies
  • 1 kudos
Latest Reply
Vidula
Honored Contributor
  • 1 kudos

Hi @Prakash Thavamurugan​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...

  • 1 kudos
1 More Replies
sameer_gupta
by New Contributor
  • 2049 Views
  • 3 replies
  • 0 kudos

Error in importing feature_store

from databricks import feature_storeI am trying to import feature_store but it is showing this error.ImportError: cannot import name 'feature_store' from 'databricks' (/databricks/python/lib/python3.8/site-packages/databricks/__init__.py)

  • 2049 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Is this issue resolved completely? We are facing the same problem. this might help.

  • 0 kudos
2 More Replies
Benji
by New Contributor II
  • 5464 Views
  • 5 replies
  • 0 kudos

Error when running job in databricks

Hello, I am very new with databricks and MLflow. I faced with the problem about running job. When the job is run, it usually failed and retried itself, so it incasesed running time, i.e., from normally 6 hrs to 12-18 hrs. From the error log, it shows...

Fail image
  • 5464 Views
  • 5 replies
  • 0 kudos
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hey there @Tanawat Benchasirirot​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hea...

  • 0 kudos
4 More Replies
jdigiovanni
by New Contributor
  • 1557 Views
  • 3 replies
  • 0 kudos

EOFError trying to assign a model using a custom module

I'm in a Data Science Bootcamp, and the final case study includes data preprocessing (done), using a linear regression model on the data, then porting to SQL for visualization. The model build uses custom python code provided as part of the exercise....

  • 1557 Views
  • 3 replies
  • 0 kudos
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Joe DiGiovanni​ Just wanted to check in if you were able to resolve your issue or do you need more help? We'd love to hear from you.Thanks!

  • 0 kudos
2 More Replies
sameer_gupta
by New Contributor
  • 2277 Views
  • 2 replies
  • 0 kudos

Error in importing mlflow.sklearn

ImportError: cannot import name '_MIN_SKLEARN_VERSION' from 'mlflow.sklearn.utils' (/databricks/python/lib/python3.8/site-packages/mlflow/sklearn/utils.py)

  • 2277 Views
  • 2 replies
  • 0 kudos
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Sameer Gupta​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...

  • 0 kudos
1 More Replies
fsyshawn
by New Contributor II
  • 1167 Views
  • 2 replies
  • 0 kudos

How can we automate MLFLOW model serving in databricks?

Can we enable model serving either using cli or any other tools without go to the databricks model UI?

  • 1167 Views
  • 2 replies
  • 0 kudos
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Shawn Feng​ Does @Atanu Sarkar​  response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!

  • 0 kudos
1 More Replies
Somi
by New Contributor III
  • 7377 Views
  • 10 replies
  • 0 kudos

Resolved! How to set sparkTrials? I am receiving this TypeError: cannot pickle '_thread.lock' object

I am trying to distribute hyperparameter tuning using hyperopt on a tensorflow.keras model. I am using sparkTrials in my fmin:spark_trials = SparkTrials(parallelism=4)...best_hyperparam = fmin(fn=CNN_HOF,                  space=space,                ...

  • 7377 Views
  • 10 replies
  • 0 kudos
Latest Reply
Dooley
Valued Contributor II
  • 0 kudos

This can happen when you try to serialize a keras model with an unserializable layer. What does your model look like? Also what is in that search space variable? What are you trying to optimize on?

  • 0 kudos
9 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels