cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

invalidargument
by New Contributor III
  • 927 Views
  • 1 replies
  • 0 kudos

Model storage requirements management

Hi.We have around 30 models in model storage that we use for batch scoring. These are created at different times by different person and on different cluster run times.Now we have run into problems that we can't de-serialize the models and use for in...

  • 927 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Jonas Lindberg​ :To address the issues you are facing with model serialization and versioning, I would recommend the following approach:Use MLflow to manage the lifecycle of your models, including versioning, deployment, and monitoring. MLflow is an...

  • 0 kudos
anvil
by New Contributor II
  • 2640 Views
  • 3 replies
  • 4 kudos

Are UDFs necessary for applying models from ML libraries at scale ?

Hello,I recently finished the "scalable machine learning with apache spark" course and saw that SKLearn models could be applied faster in a distributed manner when used in pandas UDFs or with mapInPandas() method. Spark MLlib models don't need this k...

  • 2640 Views
  • 3 replies
  • 4 kudos
Latest Reply
Manoj12421
Valued Contributor II
  • 4 kudos

MlLib is in the maintenance model and udf is not used by creating model in most cases

  • 4 kudos
2 More Replies
ianchenmu
by New Contributor II
  • 4202 Views
  • 5 replies
  • 7 kudos

Parallelization in training machine learning models using MLFlow

I'm training a ML model (e.g., XGboost) and I have a large combination of 5 hyperparameters, say each parameter has 5 candidates, it will be 5^5 = 3,125 combos.Now I want to do parallelization for the grid search on all the hyperparameter combos for ...

  • 4202 Views
  • 5 replies
  • 7 kudos
Latest Reply
Anonymous
Not applicable
  • 7 kudos

Hi @Chen Mu​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 7 kudos
4 More Replies
TomasP
by New Contributor III
  • 1848 Views
  • 3 replies
  • 0 kudos

Two or more different ml model on one cluster.

Hi, have you already dealt with the situation that you would like to have two different ml models in one cluster? i.e: I have a project which contains two or more different models with more different pursposes. The goals is to have three differ...

  • 1848 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Tomas Peterek​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 0 kudos
2 More Replies
Direo
by Contributor
  • 1684 Views
  • 2 replies
  • 1 kudos

Is it possible to load MLFlow artifacts and models from local diretory to databricks DBFS?

I have been working locally and created a few models and now I want to move those to databricks/DBFS. Is it possible to do that?

  • 1684 Views
  • 2 replies
  • 1 kudos
Latest Reply
Vivian_Wilfred
Databricks Employee
  • 1 kudos

Hi @Direo Direo​, can you check these docs and see if it helps-https://docs.databricks.com/applications/mlflow/access-hosted-tracking-server.html#access-the-mlflow-tracking-server-from-outside-databrickshttps://docs.databricks.com/applications/mlflow...

  • 1 kudos
1 More Replies
thib
by New Contributor III
  • 3299 Views
  • 4 replies
  • 2 kudos

Resolved! Feature Store : for sklearn flavored models, are timestamps fully supported?

I have created a feature table (Databricks runtime ML 10.2) that includes a timestamp column as a primary key, that is not used as a feature but as a column to join on.I have then created a model that trains from this feature table and some additiona...

  • 3299 Views
  • 4 replies
  • 2 kudos
Latest Reply
thib
New Contributor III
  • 2 kudos

Hi, it did not, but at least I know they are not fully supported so a workaround is to avoid timestamps, so I suppose you can mark this as resolved

  • 2 kudos
3 More Replies
Labels