cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Orianh
by Valued Contributor II
  • 1515 Views
  • 1 replies
  • 2 kudos

MLflow log pytorch distributed training

Hey Guys,I have few question that i hope you can help me with.I start to train pytorch model in distributed training using petastorm + Horovod like databricks suggest in docs.Q 1:I can see that each worker is train the model, but when epochs are done...

  • 1515 Views
  • 1 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@orian hindi​ :Regarding your questions:Q1: The error message you are seeing is likely related to a segmentation fault, which can occur due to various reasons such as memory access violations or stack overflows. It could be caused by several factors,...

  • 2 kudos
thibault
by Contributor
  • 1401 Views
  • 2 replies
  • 2 kudos

Best practice for model promotion so that models are not removed from previous stage

Hi,Using Model Registry to promote models is great. However, I am facing an issue, where multiple Databricks workspaces (SIT / UAT / Prod) use a model at various stages (Staging for SIT and UAT, Production for Prod workspace).We have a workflow runni...

  • 1401 Views
  • 2 replies
  • 2 kudos
Latest Reply
User16502773013
New Contributor III
  • 2 kudos

Hello Thibault,For reusing already built model there are multiple options:Register the model from dev to QA model registry as described here ORIn this scenario only the registered model will be copiedLineage to runs is not possibleYou can copy dev's ...

  • 2 kudos
1 More Replies
espenol
by New Contributor III
  • 905 Views
  • 1 replies
  • 0 kudos

Why is mounting storage no longer considered best practice?

As the title describes. I think it's really nice to work with mounted storage, but I've typically had an IaC team take care of setting it up. Now I'm not that lucky. Why is it no longer best practice? Security reasons?

  • 905 Views
  • 1 replies
  • 0 kudos
Latest Reply
xiangzhu
Contributor
  • 0 kudos

I think so, mount is like a local storage, other users in the same workspace will have the access to any mounted storage too.Access Azure Data Lake Storage Gen2 and Blob Storage | Databricks on AWS

  • 0 kudos
User16789201666
by Contributor II
  • 545 Views
  • 0 replies
  • 0 kudos

What's a best practice for Hyperopt workflow?

Choose what hyperparameters are reasonable to optimizeDefine broad ranges for each of the hyperparameters (including the default where applicable)Run a small number of trialsObserve the results in an MLflow parallel coordinate plot and select the run...

  • 545 Views
  • 0 replies
  • 0 kudos
User16826993440
by New Contributor III
  • 1994 Views
  • 1 replies
  • 1 kudos

What is the best practice for applying MLFlow to clustering algorithms?

What is the best practice for applying MLFlow to clustering algorithms? What are the kinds of metrics customers track?

  • 1994 Views
  • 1 replies
  • 1 kudos
Latest Reply
Joseph_B
New Contributor III
  • 1 kudos

Good question! I'll divide my suggestions into 2 parts:(1) In terms of MLflow Tracking, clustering is pretty similar to other ML workflows, so not much changes.(2) In terms of specific parameters, metrics, etc. to track, clustering is very different...

  • 1 kudos
Anonymous
by Not applicable
  • 1026 Views
  • 1 replies
  • 0 kudos

Resolved! Best practice for Image manipulation

Can you please recommend suggestions for image manipulation once you read the data as an image ? Any specific library to use?

  • 1026 Views
  • 1 replies
  • 0 kudos
Latest Reply
sean_owen
Honored Contributor II
  • 0 kudos

Spark has a built-in 'image' data source which will read a directory of images files as a DataFrame: spark.read.format("image").load(...). The resulting DataFrame has the pixel data, dimensions, channels, etc.You can also read image files 'manually' ...

  • 0 kudos
Labels