- 51 Views
- 1 replies
- 0 kudos
Can Databricks Jobs Run on Kubernetes Clusters?
Context: We're exploring using Kubernetes (EKS) as our compute infrastructure instead of Databricks managed clusters. We want to understand if Databricks can orchestrate, deploy, and monitor jobs that run on a Kubernetes cluster.Questions:Is it possi...
- 51 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @ASH1243434 ,Unfortunately, the Databricks cannot natively route job execution into your EKS cluster. There is no "external compute" or "bring your own Kubernetes" option in Databricks Jobs configuration. If my answer was helpful, please consider ...
- 0 kudos
- 128 Views
- 1 replies
- 0 kudos
AutoML on Azure Databricks as of June 2026
Hello everybody,My team and I are facing a sudden and unexpected issue with training Forecasting models utilizing AutoML on Azure Databricks. As of two weeks ago, we privatized our Azure Databricks environment behind a private network, accessible onl...
- 128 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @nepiskopos ,I think that your issues all share a single root cause: the restricted egress network policy introduced when you moved behind the private network.Serverless forecasting requires outbound access to resolve the workspace URL for MLflow ...
- 0 kudos
- 333 Views
- 1 replies
- 2 kudos
Resolved! Models failing in tutorial
Hello,I am following the "Get started: Build your first machine learning model on Databricks" tutorial, and am getting stuck on "Parallel training using Optuna".When I Search runs to retrieve the best model, the following code fails as there are no m...
- 333 Views
- 1 replies
- 2 kudos
- 2 kudos
You can change the objective trial code to use optuna & follow the other steps in the tutorial & run the full code seamlessly.Modify objective trial - use optuna in Free edition for serverless auth accommodation.def objective(trial): # Enable autol...
- 2 kudos
- 202 Views
- 1 replies
- 0 kudos
Whether MCP server support for Genie is available in our workspace/region for free version.
We are building a production support agent that requires seamless integration between:Genie spaces (for analytics queries)Vector search Custom tools The MCP protocol provides a standardized interface for this multi-tool architecture.i am using free t...
- 202 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Anvy, This depends on whether you are on a standard Databricks workspace or on Databricks Free Edition. The public docs show that Databricks managed MCP servers are currently in Public Preview, and the managed Genie MCP server itself is still lis...
- 0 kudos
- 832 Views
- 3 replies
- 1 kudos
Resolved! AutoML on Databricks as of May 2026
On Azure Databricks, I have been creating AutoML experiments on May 15th 2026, using a Single User compute with Databricks runtime 17.3 LTS for ML, which I have assigned to myself.I try to register one of the trained ML models in a schema, under a un...
- 832 Views
- 3 replies
- 1 kudos
- 1 kudos
Update: Today, May 19th 2026, the issue seems to have been resolved.I suppose some bug fix has been released.
- 1 kudos
- 3032 Views
- 3 replies
- 1 kudos
How to use Databricks secrets on MLFlow conda dependencies?
Hi!Do you know if it's correct to use the plain user and token for installing a custom dependency (an internal python package) in a mlflow registered model? (it's the only way I get it working because if not it can't install the dependency) It works,...
- 3032 Views
- 3 replies
- 1 kudos
- 1 kudos
Did someone solve this? I'm currently forced to download some libraries using an artifactory pypi mirror. However, I wouldn't want to have my secrets pasted in the conda.yaml file as plain text.
- 1 kudos
- 16541 Views
- 7 replies
- 3 kudos
Resolved! How to PREVENT mlflow's autologging from logging ALL runs?
I am logging runs from jupyter notebook. the cells which has `mlflow.sklearn.autlog()` behaves as expected. but, the cells which has .fit() method being called on sklearn's estimators are also being logged as runs without explicitly mentioning `mlflo...
- 16541 Views
- 7 replies
- 3 kudos
- 3 kudos
Good question—mlflow autologging can easily capture more runs than expected if not configured properly. Managing it carefully improves experiment tracking. Similar control and optimization are important in bussid mod workflows, where users fine-tune ...
- 3 kudos
- 650 Views
- 2 replies
- 0 kudos
Memory error in LightGBM training data processing
I am developing a LightGBM model on Databricks, and I am using the Native API because it offers the widest range of options and allows me to try various approaches.The training data is loaded from a table in the Catalog as a Spark DataFrame. However,...
- 650 Views
- 2 replies
- 0 kudos
- 1654 Views
- 2 replies
- 1 kudos
Resolved! Which types of model serving endpoints have health metrics available?
I am retrieving a list of model serving endpoints for my workspace via this API: https://docs.databricks.com/api/workspace/servingendpoints/listAnd then going to retrieve health metrics for each one with: https://[DATABRICKS_HOST]/api/2.0/serving-end...
- 1654 Views
- 2 replies
- 1 kudos
- 1 kudos
Your observation is correct—this behavior is expected.Endpoints with entity_type = FOUNDATION_MODEL_API do not expose health metrics via the /metrics endpoint, which is why you’re getting 404 responses. These endpoints are fully managed, multi-tenant...
- 1 kudos
- 863 Views
- 1 replies
- 0 kudos
Resolved! AWS GovCloud Feature Availability Question
Hi! I'm trying to determine if Mosaic Vector Search (or is it simply called Vector Search) is available on AWS GovCloud?This shows it is not: https://docs.databricks.com/aws/en/resources/feature-region-supportAnd it's not mentioned here: https://docs...
- 863 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @MattBuck ,It's not available on AWS GovCloud.1) The first link you attached is the authoritative source for feature availability by region. If you can't find it there it means the feature is not available in specific region2) And I think this lim...
- 0 kudos
- 808 Views
- 1 replies
- 0 kudos
Resolved! Recommended Python UDFs for On-Demand Feature Computation in Databricks
The Databricks documentation page on on-demand feature computation (https://docs.databricks.com/aws/en/machine-learning/feature-store/on-demand-features#what-are-on-demand-features) mentions using Python UDFs for computing on-demand features. What ty...
- 808 Views
- 1 replies
- 0 kudos
- 0 kudos
HI @nb92 , Only Scalar Python UDFs are allowed for on-demand feature computation. This page provides the recommended approach. Best regards,
- 0 kudos
- 1666 Views
- 7 replies
- 3 kudos
MLFlow Detailed Trace view doesn't work in some workspaces
I've created a Databricks Model Serving Endpoint which serves an MLFlow Pyfunc model. The model uses langchain and I'm using mlflow.langchain.autolog().At my company we have some production(-like) workspaces where users cannot e.g. run Notebooks and ...
- 1666 Views
- 7 replies
- 3 kudos
- 3 kudos
Funnily enough, the problem also disappeard on my end this morning Previously, I saw a networking issue in my logs, but that also went away. Let's hope it stays that way!
- 3 kudos
- 4808 Views
- 1 replies
- 0 kudos
Resolved! Using Qwen with vLLM
There are many conflict and dependency issues when trying to install VLLM and use the Qwen models (on serverless), even the v2 families.I tried following this guide https://docs.databricks.com/aws/en/machine-learning/sgc-examples/tutorials/sgc-raydat...
- 4808 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @pfzoz -- the "Model architectures failed to be inspected" error you are hitting is a well-known compatibility issue between vLLM, the transformers library, and the Qwen2/2.5-VL model family. The root cause is that vLLM's model registry subprocess...
- 0 kudos
- 1614 Views
- 3 replies
- 0 kudos
Resolved! TrainingArguments fails
Hello,I am working on an ML project for text classification and I have a problem.The following piece of code stalls completely. It prints 'start' but never 'end'.from transformers import TrainingArguments print("start") args = TrainingArguments(outpu...
- 1614 Views
- 3 replies
- 0 kudos
- 0 kudos
Hello @lingareddy_Alva ,Thank you for your reply. I have since been given a cluster with the ML Runtime and the code now works. So I consider the problem solved.
- 0 kudos
- 2648 Views
- 2 replies
- 0 kudos
Identity Resolution
Looking for best solutions for identity resolution. I already have deterministic matching. Exploring probabilistic solutions. Any advice for me?
- 2648 Views
- 2 replies
- 0 kudos
- 0 kudos
Check open source Zingg which runs natively within Databricks https://github.com/zinggAI/zingg
- 0 kudos
