- 1443 Views
- 1 replies
- 0 kudos
Serving Endpoint Deployment
Hello Community,I am seeking assistance with an issue related to serving a custom Hugging Face model (M2M100). I successfully registered the model in Unity Catalog using the MLflow Python libraries without any problems. However, when attempting to se...
- 1443 Views
- 1 replies
- 0 kudos
- 0 kudos
see https://community.databricks.com/t5/machine-learning/serving-endpoint-container-image-creation-fails/td-p/83458
- 0 kudos
- 1926 Views
- 2 replies
- 0 kudos
Azure databricks API and DLT databricks
how can i pass parameter from Azure data factory rest web API to delta live Databricks?I get this error: "Py4JJavaError: An error occurred while calling o382.getArgument.: com.databricks.dbutils_v1.InputWidgetNotDefined: No input widget named *** def...
- 1926 Views
- 2 replies
- 0 kudos
- 0 kudos
@girl_chan The error you are encountering usually occurs when a widget referenced in a Databricks notebook is not defined. In your case, the issue is likely related to how you pass parameters from Azure Data Factory (ADF) to Delta Live Tables (DLT) i...
- 0 kudos
- 2802 Views
- 2 replies
- 0 kudos
Rollback cluster changes
Is it possible to rollback changes made to a cluster? The problem I'm trying to solve is to recover from an accidental change made by a user on a cluster that affects interactive and job runs. Cluster policies help, but the policy still provides the ...
- 2802 Views
- 2 replies
- 0 kudos
- 0 kudos
@User16826990884 Along with what @sajith_appukutt mentioned, we can achive this viaVersion Control for Cluster Configurations: Store cluster configurations in JSON files in GitHub or another version control system.In case of accidental changes, you c...
- 0 kudos
- 3501 Views
- 0 replies
- 0 kudos
Custom AutoML pipeline: Beyond StandardScaler().
The automated notebook pipeline in an AutoML experiment applies StandardScaler to all numerical features in the training dataset as part of the PreProcessor. See below.But I want a more nuanced and varied treatment of my numeric values (e.g. I have l...
- 3501 Views
- 0 replies
- 0 kudos
- 3286 Views
- 0 replies
- 0 kudos
Surprisingly sparse_logs and tensorboard logfiles in Databricks-Workspace
Hi, surprisingly we have found 2 new folders with some short logfiles in our Databricks workspace:ls -lFr sparse_logs/ tensorboard/tensorboard/:-rwxrwxrwx 1 root root 88 Sep 2 11:26 events.out.tfevents.1725275744.0830-063833-n68nsxoq-10-139-64-10.20...
- 3286 Views
- 0 replies
- 0 kudos
- 4413 Views
- 2 replies
- 1 kudos
What is the best practice for applying MLFlow to clustering algorithms?
What is the best practice for applying MLFlow to clustering algorithms? What are the kinds of metrics customers track?
- 4413 Views
- 2 replies
- 1 kudos
- 1 kudos
Good question! I'll divide my suggestions into 2 parts:(1) In terms of MLflow Tracking, clustering is pretty similar to other ML workflows, so not much changes.(2) In terms of specific parameters, metrics, etc. to track, clustering is very different...
- 1 kudos
- 3685 Views
- 0 replies
- 1 kudos
AutoML Doesn't Work Due to Not being able to generate the EDA notebook
HiI'm trying run AutoML classification experiment with a dataset that I have made, and am experiencing this issue even after I have purposely downsampled my dataset before running it into the AutoML experiment. It appears that there is no way for me ...
- 3685 Views
- 0 replies
- 1 kudos
- 3219 Views
- 2 replies
- 1 kudos
Machine Learning with Databricks Lab Notebooks
I'm working through this course and the training video says I should have access to the demo notebook. I do not - nor can I locate it or a link to it anywhere. Github says access to the notebooks is no longer on Github and links me back to the cust...
- 3219 Views
- 2 replies
- 1 kudos
- 1 kudos
Ya, I don't see anything like that. Maybe it's just not available for the customer academy. I received a response from Databricks by email, but it wasn't helpful at all. They really need to provide more information about whether I should have acce...
- 1 kudos
- 1429 Views
- 1 replies
- 0 kudos
Populate client_request_id in Model Serving inference table
Hi,The documentation for the model serving inference table states that the client_request_id column is typically null. How can I populate this column with a request ID from the calling .NET application when invoking the model via the Databricks REST ...
- 1429 Views
- 1 replies
- 0 kudos
- 0 kudos
I finally found it in the docs: https://docs.databricks.com/en/machine-learning/model-serving/inference-tables.html#specify-client_request_id { "client_request_id": "<user-provided-id>", "dataframe_records": [...]}
- 0 kudos
- 5595 Views
- 4 replies
- 1 kudos
Databricks MlFlow Error: Timed out while evaluating the model.
Hi everyone,I am using databricks and mlflow to create a model and then register it as a serving endpoint. Sometimes the models takes more than 2 minutes to run and after 2 minutes it gives a timeout error:Timed out while evaluating the model. Verify...
- 5595 Views
- 4 replies
- 1 kudos
- 41052 Views
- 2 replies
- 1 kudos
Resolved! torch.cuda.OutOfMemoryError: CUDA out of memory
Hi,I am using pynote/whisper large model and trying to process data using spark UDF and getting following error.torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 172.00 MiB (GPU 0; 14.76 GiB total capacity; 6.07 GiB already allocated...
- 41052 Views
- 2 replies
- 1 kudos
- 1 kudos
Try to run these codesimport torchtorch.cuda.empty_cache()And make sure to find the optimize batch size otherwise the error can occur again
- 1 kudos
- 1133 Views
- 2 replies
- 0 kudos
Error on Workflow: Failure to initialize configuration for storage account
I have set up a workflow with a sequence of jobs. Each job run fine in an interactive mode, that is, run the notebook directly. However, when I tried to run the workflow, it got error on a step which uses a function from a Repo. the error says "Failu...
- 1133 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Tingting ,It seems that when you run notebook interactively, your personal credentials are used to access ADLS.When the workflow job is run, Databricks uses different context. Could you share whether your job is accessing some storage account, an...
- 0 kudos
- 1390 Views
- 1 replies
- 0 kudos
Resolved! Mlflow not saving flavor correctly
Hello!Im trying to save my model with mlflow in databricks, it is a xgboost model, when I save it using code it saves with a sklearn flavor and not saves other parameters, also I'm using kedro with kedro-mlflow plugin.def log_metrics_and_model(model,...
- 1390 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello!It was the magic of all porpoise clusters, just restart the cluster and done x.x
- 0 kudos
- 3646 Views
- 0 replies
- 0 kudos
Model serving with custom pip index URL
An mlflow model was logged with a custom pip requirements file which contains package versions (mlflow==2.11.3), as well as a custom --index-url. However model serving during the "Initializing model enviroment" step tries to pip install mlflow==2.2.2...
- 3646 Views
- 0 replies
- 0 kudos
- 3365 Views
- 0 replies
- 0 kudos
Error to create an endpoint of databricks with 2 primary keys online table
I have a delta table that has a primary key conformed by 2 fields (accountId,ruleModelVersionDesc) and I have also created an online table that has the same primary key, but when I create a feature spec to create an endpoint I get the following error...
- 3365 Views
- 0 replies
- 0 kudos
