- 3253 Views
- 4 replies
- 2 kudos
Best practice for model promotion so that models are not removed from previous stage
Hi,Using Model Registry to promote models is great. However, I am facing an issue, where multiple Databricks workspaces (SIT / UAT / Prod) use a model at various stages (Staging for SIT and UAT, Production for Prod workspace).We have a workflow runni...
- 3253 Views
- 4 replies
- 2 kudos
- 2135 Views
- 1 replies
- 2 kudos
MLflow log pytorch distributed training
Hey Guys,I have few question that i hope you can help me with.I start to train pytorch model in distributed training using petastorm + Horovod like databricks suggest in docs.Q 1:I can see that each worker is train the model, but when epochs are done...
- 2135 Views
- 1 replies
- 2 kudos
- 2 kudos
@orian hindi​ :Regarding your questions:Q1: The error message you are seeing is likely related to a segmentation fault, which can occur due to various reasons such as memory access violations or stack overflows. It could be caused by several factors,...
- 2 kudos
- 1274 Views
- 1 replies
- 0 kudos
Why is mounting storage no longer considered best practice?
As the title describes. I think it's really nice to work with mounted storage, but I've typically had an IaC team take care of setting it up. Now I'm not that lucky. Why is it no longer best practice? Security reasons?
- 1274 Views
- 1 replies
- 0 kudos
- 0 kudos
I think so, mount is like a local storage, other users in the same workspace will have the access to any mounted storage too.Access Azure Data Lake Storage Gen2 and Blob Storage | Databricks on AWS
- 0 kudos
- 781 Views
- 0 replies
- 0 kudos
What's a best practice for Hyperopt workflow?
Choose what hyperparameters are reasonable to optimizeDefine broad ranges for each of the hyperparameters (including the default where applicable)Run a small number of trialsObserve the results in an MLflow parallel coordinate plot and select the run...
- 781 Views
- 0 replies
- 0 kudos
- 2413 Views
- 1 replies
- 1 kudos
What is the best practice for applying MLFlow to clustering algorithms?
What is the best practice for applying MLFlow to clustering algorithms? What are the kinds of metrics customers track?
- 2413 Views
- 1 replies
- 1 kudos
- 1 kudos
Good question! I'll divide my suggestions into 2 parts:(1) In terms of MLflow Tracking, clustering is pretty similar to other ML workflows, so not much changes.(2) In terms of specific parameters, metrics, etc. to track, clustering is very different...
- 1 kudos
- 1367 Views
- 1 replies
- 0 kudos
Resolved! Best practice for Image manipulation
Can you please recommend suggestions for image manipulation once you read the data as an image ? Any specific library to use?
- 1367 Views
- 1 replies
- 0 kudos
- 0 kudos
Spark has a built-in 'image' data source which will read a directory of images files as a DataFrame: spark.read.format("image").load(...). The resulting DataFrame has the pixel data, dimensions, channels, etc.You can also read image files 'manually' ...
- 0 kudos