Hi all,I'm configuring a Model Serving endpoint with two served entities and ran into some unexpected behavior while testing different traffic_config splits.When I set traffic_percentage to 100 for each of the two served entities (so the total sums t...
I am doing a case choice analysis on whether I should have my data in Snowflake to support my Customer workloads or have them migrated to Databricks to do the same.What has been your experience especially in handling large volumes of data especially ...
Given the pace in the new AI era, which platform have the most inovations that brings value to customers clearly wins it. And we can see Databricks is releasing new features on weekly basis, they are clearly not behind.
In enterprise environments, the bottleneck of Machine Learning is rarely the algorithms themselves—it is the engineering around them. Data scientists often spend days configuring environments, managing infrastructure, and manually tracking model iter...
Hi everyone,I'm working on a personal project and would appreciate some advice from people who have experience with machine learning on Databricks.Imagine having a dataset containing thousands of game compatibility records collected from different em...
This is a classification problem, not a recommendation system. Recommenders are for "which item does this user prefer among many," but what you're describing is "does this game run on this config," which is a label you're predicting from a se...
Dear Community I need some support with investigation related to Serving Endpoints. Recently some of endpoints with deployed ML models display message:This endpoint is out of compliance because it is too old and automatic system updates have failed.P...
Hi, Could you provide a workspaceID and endpointID and any other details such as screenshots? Will check to see what I can find based on those details, ~Mo.
When trying to use serverless gpu on a notebook, I get this error. Serverless CPU works fine though.Can you help me understand why I get this?
Hi @LPBigot, Since Serverless CPU is working but Serverless GPU is failing when the notebook tries to attach compute, could you confirm a couple of things so we can narrow this down? Is this a paid/Premium+ workspace, or is it a free/trial-style setu...
Hi all,I am getting the following error when using CrossValidator in Databricks Free Edition v5 (see attachment):In shared or serverless cluster, SPARK_ML_TMP_DFS_PATH environmental variable must be set to a UC volume path like '/Volumes/...' in orde...
What you are encountering is an expected behavior on Databricks serverless. Because CrossValidator (along with other Spark ML tuning estimators) relies heavily on internal DataFrame caching to optimize iterative model training, Databricks enforces st...
Hi!Do you know if it's correct to use the plain user and token for installing a custom dependency (an internal python package) in a mlflow registered model? (it's the only way I get it working because if not it can't install the dependency) It works,...
The dynamic conda.yaml generation trick doesn't actually fix this, and you already found that out yourself. Whatever ends up in the final logged conda.yaml is what gets read at environment build time, so if a plaintext credential has to be in that fi...
Hello everybody,My team and I are facing a sudden and unexpected issue with training Forecasting models utilizing AutoML on Azure Databricks. As of two weeks ago, we privatized our Azure Databricks environment behind a private network, accessible onl...
Databricks AutoML is still around and actively used as of mid-2026, so nothing's wrong on that front, but there's a nuance worth knowing: starting with Databricks Runtime 18.0 ML, AutoML was removed as a built-in library on classic compute. It's stil...
Hello can someone tell me where do I find the notebooks in the demo sessions of Associate ML pathways?
Hi @ShaRKS, If you are taking the Associate ML pathway, the notebooks are typically available through the course's lab environment rather than as separate downloadable assets. Databricks course pages for the ML curriculum note that lecture notebooks ...
what is the GPU quota limit for free edition? I used 5 minutes few days back now can not attach GPU cluster, it showing quota exceeded
I have opened the 14 days trial version , still not able to do this .. have not used any compute yet..
Hello,I'm trying to set up a DAB job that runs an ML job. For this it would be useful to use a serverless ML environment, that I can select in notebooks. Anyway, I do not find a meaningful way to define the base environment as ML.I do not want to giv...
You can create the environment in the DABenvironments: - environment_key: default spec: environment_version: "5" dependencies: - xgboost # AddYou can also create a file with packag...
Context: We're exploring using Kubernetes (EKS) as our compute infrastructure instead of Databricks managed clusters. We want to understand if Databricks can orchestrate, deploy, and monitor jobs that run on a Kubernetes cluster.Questions:Is it possi...
Hi @ASH1243434 ,Unfortunately, the Databricks cannot natively route job execution into your EKS cluster. There is no "external compute" or "bring your own Kubernetes" option in Databricks Jobs configuration. If my answer was helpful, please consider ...
Hello,I am following the "Get started: Build your first machine learning model on Databricks" tutorial, and am getting stuck on "Parallel training using Optuna".When I Search runs to retrieve the best model, the following code fails as there are no m...
You can change the objective trial code to use optuna & follow the other steps in the tutorial & run the full code seamlessly.Modify objective trial - use optuna in Free edition for serverless auth accommodation.def objective(trial): # Enable autol...
We are building a production support agent that requires seamless integration between:Genie spaces (for analytics queries)Vector search Custom tools The MCP protocol provides a standardized interface for this multi-tool architecture.i am using free t...
Hi @Anvy, This depends on whether you are on a standard Databricks workspace or on Databricks Free Edition. The public docs show that Databricks managed MCP servers are currently in Public Preview, and the managed Genie MCP server itself is still lis...
