by
Orianh
• Valued Contributor II
- 1312 Views
- 1 replies
- 2 kudos
Hey Guys,I have few question that i hope you can help me with.I start to train pytorch model in distributed training using petastorm + Horovod like databricks suggest in docs.Q 1:I can see that each worker is train the model, but when epochs are done...
- 1312 Views
- 1 replies
- 2 kudos
Latest Reply
@orian hindi​ :Regarding your questions:Q1: The error message you are seeing is likely related to a segmentation fault, which can occur due to various reasons such as memory access violations or stack overflows. It could be caused by several factors,...
- 1149 Views
- 2 replies
- 2 kudos
Hi,Using Model Registry to promote models is great. However, I am facing an issue, where multiple Databricks workspaces (SIT / UAT / Prod) use a model at various stages (Staging for SIT and UAT, Production for Prod workspace).We have a workflow runni...
- 1149 Views
- 2 replies
- 2 kudos
Latest Reply
Hello Thibault,For reusing already built model there are multiple options:Register the model from dev to QA model registry as described here ORIn this scenario only the registered model will be copiedLineage to runs is not possibleYou can copy dev's ...
1 More Replies
- 786 Views
- 1 replies
- 0 kudos
As the title describes. I think it's really nice to work with mounted storage, but I've typically had an IaC team take care of setting it up. Now I'm not that lucky. Why is it no longer best practice? Security reasons?
- 786 Views
- 1 replies
- 0 kudos
Latest Reply
I think so, mount is like a local storage, other users in the same workspace will have the access to any mounted storage too.Access Azure Data Lake Storage Gen2 and Blob Storage | Databricks on AWS
- 495 Views
- 0 replies
- 0 kudos
Choose what hyperparameters are reasonable to optimizeDefine broad ranges for each of the hyperparameters (including the default where applicable)Run a small number of trialsObserve the results in an MLflow parallel coordinate plot and select the run...
- 495 Views
- 0 replies
- 0 kudos
- 1856 Views
- 1 replies
- 1 kudos
What is the best practice for applying MLFlow to clustering algorithms? What are the kinds of metrics customers track?
- 1856 Views
- 1 replies
- 1 kudos
Latest Reply
Good question! I'll divide my suggestions into 2 parts:(1) In terms of MLflow Tracking, clustering is pretty similar to other ML workflows, so not much changes.(2) In terms of specific parameters, metrics, etc. to track, clustering is very different...
- 953 Views
- 1 replies
- 0 kudos
Can you please recommend suggestions for image manipulation once you read the data as an image ? Any specific library to use?
- 953 Views
- 1 replies
- 0 kudos
Latest Reply
Spark has a built-in 'image' data source which will read a directory of images files as a DataFrame: spark.read.format("image").load(...). The resulting DataFrame has the pixel data, dimensions, channels, etc.You can also read image files 'manually' ...
