- 337 Views
- 1 replies
- 0 kudos
workflow not pickingup correct host value (While working with MLflow model registry URI)
Exception: mlflow.exceptions.MlflowException: An API request to https://canada.cloud.databricks.com/api/2.0/mlflow/model-versions/list-artifacts failed due to a timeout. The error message was: HTTPSConnectionPool(host='canada.cloud.databricks.com', p...
- 337 Views
- 1 replies
- 0 kudos
- 0 kudos
@Dharma25 The error clearly shows that MLflow requests are being incorrectly routed to "canada.cloud.databricks.com" instead of your actual workspace URL, causing the timeout.This is a known issue that can occur with MLflow in Databricks, particularl...
- 0 kudos
- 1748 Views
- 3 replies
- 0 kudos
Resolved! Error in creating a serving endpoint: registered model not found
I have registered a custom model which loads another model in the load_context method. Everything works fine when I load (with mlflow.pyfunc.load_model) and use the model in a notebook. When I try to create a serving endpoint for it I keep becoming t...
- 1748 Views
- 3 replies
- 0 kudos
- 0 kudos
It is registered in the Unity Catalog. I have found a complete other solution now. With the help of TransformedTargetRegressor I don't need a separate normalisation step anymore and therefore don't load a model in load_context anymore.
- 0 kudos
- 98 Views
- 3 replies
- 1 kudos
Resolved! How to install Tensorflow 1 based compute or packages in Databricks
I want to install Tensorflow 1 based packages along with python 3.7 etc. I tried multiple ways including using a custom docker image. But nothing seems to workAlso I know that the minimum runtime version available in Databricks is 10.4So is it possib...
- 98 Views
- 3 replies
- 1 kudos
- 1 kudos
@aswinkks You're right to be cautious — as of 2025, using TensorFlow 1.x in modern environments likeDatabricks has become increasingly difficult, if not practically unsupported, due to the combination of:- Deprecation of Python 3.7- TensorFlow 1.x be...
- 1 kudos
- 243 Views
- 1 replies
- 0 kudos
Model Serving Endpoint: Cuda-OOM for Custom Model
Hello all,I am tasked to evaluate a new LLM for some use-cases. In particular, I need to build a POC for a chat bot based on that model. To that end, I want to create a custom Serving Endpoint for an LLM pulled from huggingfaces. The model itself is...
- 243 Views
- 1 replies
- 0 kudos
- 0 kudos
Here are some suggestions: 1. Update coda.yaml. Replace the current config with this optimized version: channels: - conda-forge dependencies: - python=3.10 # 3.12 may cause compatibility issues - pip - pip: - mlflow==2.21.3 - torch...
- 0 kudos
- 700 Views
- 2 replies
- 0 kudos
Problem with ipywidgets and plotly on Databricks
Hi everyone, I am encountering a problem when using ipywidgets with plotly on Databricks. I am trying to pass interactive arguments to a function and then plot with plotly. When I do the followingdef f(m, b) : plt.figure(2) x = np.linspace(-10,...
- 700 Views
- 2 replies
- 0 kudos
- 0 kudos
Thanks for the suggestion! You're absolutely right. The code was already all in my message, but I can make it easier to copy-paste (and add the imports):from ipywidgets import interactive import matplotlib.pyplot as plt import numpy as np def f(m, b...
- 0 kudos
- 23615 Views
- 6 replies
- 6 kudos
Resolved! Access the environment variable from the custom container base cluster
Hi Databricks Community, I want to set environment variables for all clusters in my workspace. The goal is to the have environment variable, available in all notebooks executed on the cluster.The environment variable is generated in global init scrip...
- 23615 Views
- 6 replies
- 6 kudos
- 6 kudos
Thanks @Lukasz Lu​ - that worked for me as well. When I used the following script:#!/bin/bash echo MY_TEST_VAR=value1 | tee -a /etc/environment >> /databricks/spark/conf/spark-env.shfor non-docker clusters, MY_TEST_VAR shows up twice in ` /databrick...
- 6 kudos
- 2595 Views
- 2 replies
- 1 kudos
Resolved! Using Datbricks Connect with serverless compute and MLflow
Hi all,I have been using databricks-connect with serverless compute to develop and debug my databricks related code. It worked great so far. Now I started integrating ML-Flow in my workflow, and I am encountering an issue. When I run the following co...
- 2595 Views
- 2 replies
- 1 kudos
- 1 kudos
The error you are encountering, pyspark.errors.exceptions.connect.AnalysisException: [CONFIG_NOT_AVAILABLE] Configuration spark.mlflow.modelRegistryUri is not available. SQLSTATE: 42K0I, is a known issue when using MLflow with serverless clusters in ...
- 1 kudos
- 270 Views
- 0 replies
- 0 kudos
Custom Multi-agent deployment error
Hi. I am deploying a custom multi-agent system comprising of a genie agent and a RAG solution. While deploying, I am getting the following error:I am using 16.1 ML (Node: Standard_D4ads_v5 16GB,4 core) cluster and I am using the following code for lo...
- 270 Views
- 0 replies
- 0 kudos
- 383 Views
- 3 replies
- 0 kudos
Error when uploading MLFlow artifacts to DBFS
Hi everyone,I'm attempting to use MLFlow experiment tracking from a local machine, but I'm encountering difficulties in uploading artifacts.I've tried a sample code as simple as the following.import mlflow import os os.environ["DATABRICKS_HOST"] = "...
- 383 Views
- 3 replies
- 0 kudos
- 0 kudos
It is considered best practice not to store any production data or assets in DBFS (Databricks File System). The primary reason is that DBFS does not provide robust security controls-anyone with workspace access can potentially access items stored the...
- 0 kudos
- 2515 Views
- 4 replies
- 2 kudos
Problem serving a langchain model on Databricks
Hi, I've encountered a problem of serving a langchain model I just created successfully on Databricks.I was using the following code to set up a model in unity catalog:from mlflow.models import infer_signatureimport mlflowimport langchainmlflow.set_r...
- 2515 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi,The warnings/errors in the logs of the langchain model log process can give you a good hint, although it may be not that evident at first sight.It happened something similar to me - same error message, and the cause was having used an OpenAI model...
- 2 kudos
- 2117 Views
- 2 replies
- 0 kudos
Accessing Databricks Volumes from a Serving Endpoint Using a Custom Model Class in Unity Catalog
Hi everyone,I’m looking for accessing Unity Catalog (UC) Volumes from a Databricks Serving Endpoint. Here’s my current setup:I have a custom AI model class for inference, which I logged into Unity Catalog using mlflow.pyfunc.log_model.I’ve created a ...
- 2117 Views
- 2 replies
- 0 kudos
- 0 kudos
Hey VELU1122,did you find a solution for it. We are struggling with the same problem currently. Thanks
- 0 kudos
- 1334 Views
- 4 replies
- 0 kudos
Resolved! FeatureEngineeringClient workspace id error
Hi, I am working from local notebook using vscode databricks extension.I am trying to use FeatureEngineeringClient, when I create data set training_set = fe.create_training_set( df=filtered_data_train, feature_lookups=payments_feat...
- 1334 Views
- 4 replies
- 0 kudos
- 0 kudos
I’ve done some additional research and found that the FeatureStoreClient is not officially supported when accessing a managed Databricks environment from an external IDE, even when using Databricks Connect. The client library is designed to operate w...
- 0 kudos
- 2286 Views
- 2 replies
- 2 kudos
"error_code":"INVALID_PARAMETER_VALUE","message":"INVALID_PARAMETER_VALUE: Failed to generate access
Hello everyone,I have an Azure Databricks subscription with my company, and I want to use external LLMs in databricks, like claude-3 or gemini. I managed to create a serving endpoint for Anthropic and I am able to use claude 3.But I want to use a Gem...
- 2286 Views
- 2 replies
- 2 kudos
- 1668 Views
- 1 replies
- 0 kudos
Resolved! Enabled the AI Builder Preview but unable to see the feature on the menu even after 3-4 hours
I am an account admin and enabled the beta feature. Does any additional permissions need to be added before I can see the feature on the workspace.
- 1668 Views
- 1 replies
- 0 kudos
- 0 kudos
Realized that our workspace is hosted in a different region. AI Builder is available for only couple of regions at the moment. I was able to spin up a new workspace and it works. Can close this thread
- 0 kudos
- 469 Views
- 0 replies
- 0 kudos
When does everyone utilize the model register?
Hi, I'm Yuki,I'm considering when I should use register_model.In my case, I'm running the training batch once a week and if the model is good, I want to update the champion.I have created the code to register the model if the score is the best.# star...
- 469 Views
- 0 replies
- 0 kudos
