- 2172 Views
- 1 replies
- 1 kudos
Resolved! Using AutoML in Azure Databricks with a shared cluster
Do you have to use AutoML in Azure Databricks on a personal compute cluster or can you use a shared cluster?Can you point me to some documentation that supports the statement that you can run AutoML on a shared cluster with Azure Databricks?
- 2172 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @david_stroud Greetings! AutoML is not supported on Shared clusters. Please check the documentation below https://learn.microsoft.com/en-us/azure/databricks/machine-learning/automl/#--requirements
- 1 kudos
- 12603 Views
- 4 replies
- 1 kudos
Mlflowexception: "Connection broken: ConnectionResetError(104, \\\'Connection reset by peer\\\')"
Hello,I have a workflow running which from time to time crashes with the error:MlflowException: The following failures occurred while downloading one or more artifacts from models:/incubator-forecast-charging-demand-power-and-io-dk2/Production: {'pyt...
- 12603 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @cl2,Thanks for bringing up your concerns; always happy to help Upon going through the details, it appears there was an HTTP connection error downloading artifacts. This typically shouldn’t happen, but it can occur intermittently as a transient n...
- 1 kudos
- 8428 Views
- 6 replies
- 3 kudos
`collect()`ing Large Datasets in R
Background: I'm working on a pilot project to assess the pros and cons of using DataBricks to train models using R. I am using a dataset that occupies about 5.7GB of memory when loaded into a pandas dataframe. The data are stored in a delta table in ...
- 8428 Views
- 6 replies
- 3 kudos
- 3 kudos
@acsmaggart Please try using collect_larger() to collect the larger dataset. This should work. Please refer to the following document for more info on the library.https://medium.com/@NotZacDavies/collecting-large-results-with-sparklyr-8256a0370ec6
- 3 kudos
- 3160 Views
- 2 replies
- 5 kudos
How can I view the storage space taken by a registered model using MLFlow?
The information viewed about the registered models on the Models tab is very minimal. Just showing the tags we pass in and version information. How can I get more details about the model such as the size on disk?
- 3160 Views
- 2 replies
- 5 kudos
- 5 kudos
Hi,I have used the MLFlow client, but I am not sure where to find the size of the model image.The response to client.search_registered_models() I am getting is the following:<RegisteredModel: aliases={}, creation_timestamp=17061..., description='', l...
- 5 kudos
- 13271 Views
- 5 replies
- 0 kudos
Resolved! Model serving endpoint requires workspace-access entitlement?
Hi all, is anyone getting status 403 when requesting a model serving endpoint with error message "This API is disabled for users without the workspace-access entitlement"? I am accessing my model serving endpoint with a service principal access token...
- 13271 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @run480 , We understand that you are facing the following error while you are trying to access the model serving endpoint with a Service Principal Access Token: ++++++++++++++++++++++++++++++++++++++ "This API is disabled for users without the wor...
- 0 kudos
- 1846 Views
- 0 replies
- 0 kudos
Editing posts
Hey,I have a post that I would like to edit. However, when I use the drop down menu there is no possibility to edit or delete the post - can someone help?
- 1846 Views
- 0 replies
- 0 kudos
- 1853 Views
- 0 replies
- 0 kudos
Loading Pre-trained Models in Databricks
Hello talented members of the community,I'm a very new Databricks user so please bear with me.I'm building a description matcher which uses a pre-trained model (universal-sentence-encoder). How can I load and use this model in my Databricks python no...
- 1853 Views
- 0 replies
- 0 kudos
- 8055 Views
- 0 replies
- 0 kudos
Ready to elevate your data insights to new heights? Discover the power of Databricks SQL Materialize
In our latest blog post, unlock the secrets to delivering fresh data and actionable insights straight to your business. Learn how Lakeview dashboards, now available on the Databricks Data Intelligence Platform, revolutionize visualization and reporti...
- 8055 Views
- 0 replies
- 0 kudos
- 2302 Views
- 2 replies
- 0 kudos
How to use Databricks secrets on MLFlow conda dependencies?
Hi!Do you know if it's correct to use the plain user and token for installing a custom dependency (an internal python package) in a mlflow registered model? (it's the only way I get it working because if not it can't install the dependency) It works,...
- 2302 Views
- 2 replies
- 0 kudos
- 0 kudos
Thank you very much for the response but using that way is the same as the plain text approach, right? I mean, you are writing it from a Notebook and that's what I've done, but if you open the .yaml file the values are there with plain text.
- 0 kudos
- 3175 Views
- 2 replies
- 0 kudos
Update model serving endpoint
Hi all,I've been able to create a model serving endpoint through the api, following the docs, but then when trying to update the version, I get the following error:'{"error_code":"RESOURCE_ALREADY_EXISTS","message":"Endpoint with name \'ml-project\' ...
- 3175 Views
- 2 replies
- 0 kudos
- 0 kudos
Folks, Alternate way you can also deploy the models in serving layer with different versions. Though I am using mLflow. You may also refer to the below link if it its helpful How to Quickly Deploy, Test & Manage ML Models as REST Endpoints with Datab...
- 0 kudos
- 8055 Views
- 6 replies
- 0 kudos
TypeError: 'JavaPackage' object is not callable
Hi Team,I am facing issue with above error while I am trying to do BERT embeddings, by specifying the model path and it is giving error while downloading the model.spark version is 3.3.0Can any one of you help me on this?
- 8055 Views
- 6 replies
- 0 kudos
- 0 kudos
please give error detail.i think you need add some Maven package, like https://community.databricks.com/t5/machine-learning/synapse-ml-typeerror-javapackage-object-is-not-callable/td-p/58897
- 0 kudos
- 2801 Views
- 1 replies
- 0 kudos
Synapse ML - TypeError: 'JavaPackage' object is not callable
from synapse.ml.lightgbm import *LightGBMRegressor()gives me TypeError: 'JavaPackage' object is not callableDBR 8.2 (includes Apache Spark 3.1.1, Scala 2.12)
- 2801 Views
- 1 replies
- 0 kudos
- 0 kudos
You need install Maven package `com.microsoft.azure:synapseml_2.12:1.0.2`https://microsoft.github.io/SynapseML/docs/Get%20Started/Install%20SynapseML/#databricks
- 0 kudos
- 5109 Views
- 3 replies
- 0 kudos
org.apache.spark.SparkException: Job aborted due to stage failure during Model Training
org.apache.spark.SparkException: Job aborted due to stage failure: Could not recover from a failed barrier ResultStage. Most recent failure reason: Stage failed because barrier task ResultTask(160, 13) finished unsuccessfully.
- 5109 Views
- 3 replies
- 0 kudos
- 9977 Views
- 4 replies
- 2 kudos
Resolved! Error when accessing rdd of DataFrame
I need to run this kind of code: from pyspark.sql import SparkSession import pandas as pd # Create a Spark session spark = SparkSession.builder.appName("example").getOrCreate() # Sample data data = [("Alice", 1), ("Bob", 2), ("Charlie", 3), ("David...
- 9977 Views
- 4 replies
- 2 kudos
- 2 kudos
I found a good solution that works both locally and in the cloud. Copy pasting the code in case it helps someone.This is the higher level function in charge of partitioning the data and sending the data and the function fn to each node. def decrypt_d...
- 2 kudos
- 1816 Views
- 0 replies
- 0 kudos
How to use mlflow to log a composite estimator (multiple pipes) and then deploy it as rest endpoint
Hello,I am trying to deploy a composite estimator as single model, by logging the run with mlflow and registering the model.Can anyone help with how this can be done? This estimator contains different chains-text: data- tfidf- svm- svm.decision_funct...
- 1816 Views
- 0 replies
- 0 kudos
