cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

ticusss
by New Contributor
  • 141 Views
  • 1 replies
  • 1 kudos

Model Serving An internal error occurred during feature store lookup all deploys failing

We can't deploy models to Model Serving. Failures started around 2026-08-16 and were intermittent at first — our current production config deployed cleanly on 08-20 — but since then every attempt fails at the feature store lookup setup step, with no ...

  • 141 Views
  • 1 replies
  • 1 kudos
Latest Reply
AbhilashNagilla
Databricks Employee
  • 1 kudos

The upgrade documentation says existing feature and model serving workloads continue "with no action required" and the create_online_store and update_online_store interfaces need no changes. It doesn't tie Failed to fetch database instance with ID ....

  • 1 kudos
kartheek_rao
by New Contributor II
  • 344 Views
  • 3 replies
  • 2 kudos

End-to-End Streaming NLP Pipeline with GDELT, Azure Data Factory, ADLS Gen2 and Databricks

I have been working on a project to understand Databricks end to end, rather than just loading some data and training a model.I picked GDELT news data and the use case is to identify supply chain disruption related news and eventually predict which e...

  • 344 Views
  • 3 replies
  • 2 kudos
Latest Reply
ThiamLee
New Contributor II
  • 2 kudos

Really like this approach. The shift from just “data → model” to thinking about ingestion, replayability, and model lifecycle is what makes this feel like a real-world ML project.

  • 2 kudos
2 More Replies
ben_ai
by New Contributor
  • 65 Views
  • 0 replies
  • 0 kudos

Image Annotation

What is Image Annotation?What are the Steps of Image Annotation?What are the Different Techniques of Image Annotation?Types Used in Image AnnotationHow are Companies Handling Image Annotation?Features to Look for in Image Annotation Service Providers...

  • 65 Views
  • 0 replies
  • 0 kudos
airbots
by New Contributor II
  • 201 Views
  • 2 replies
  • 1 kudos

Vector Store update stale in Syncing status even the actual sync task is done.

VS status remains "syncing" indefinitely — rechecked after 12 hours, status still "syncing". The Databricks job completes successfully and the new data is queryable:

  • 201 Views
  • 2 replies
  • 1 kudos
Latest Reply
balajij8
Esteemed Contributor II
  • 1 kudos

@airbotsThe underlying issue is generally a metadata reconciliation lag where the Vector Search index status remains stuck displaying Syncing in Catalog Explorer even after the sync pipeline has completed. The metadata status reporting mechanism gene...

  • 1 kudos
1 More Replies
lucas1147
by New Contributor II
  • 429 Views
  • 2 replies
  • 2 kudos

Using Machine Learning to Improve Emulator Compatibility Predictions

Hi everyone,I'm working on a personal project and would appreciate some advice from people who have experience with machine learning on Databricks.Imagine having a dataset containing thousands of game compatibility records collected from different em...

  • 429 Views
  • 2 replies
  • 2 kudos
Latest Reply
ThiamLee
New Contributor II
  • 2 kudos

Yeah, I’d start simple with XGBoost or LightGBM and use AutoML for a quick baseline. Then just focus on the features that actually improve the results.

  • 2 kudos
1 More Replies
aswinkks
by New Contributor III
  • 452 Views
  • 4 replies
  • 7 kudos

Resolved! ML Training low File I/O and Throughout

Hi everyone,I have an image-based deep learning workload running on Azure Databricks, while the training dataset must remain in AWS S3 due to some constraints. We cannot move or replicate the dataset to Azure.Our current architecture is roughly:AWS S...

  • 452 Views
  • 4 replies
  • 7 kudos
Latest Reply
ThiamLee
New Contributor II
  • 7 kudos

I’d first check whether the bottleneck is S3/network latency or image decoding. For multi-epoch training, local caching + larger MDS shards might help, but the cross-cloud setup could still be the main issue.

  • 7 kudos
3 More Replies
MageshS
by New Contributor II
  • 545 Views
  • 1 replies
  • 1 kudos

Resolved! Traffic split behavior when traffic_percentage values across served entities sum to more than 100%

Hi all,I'm configuring a Model Serving endpoint with two served entities and ran into some unexpected behavior while testing different traffic_config splits.When I set traffic_percentage to 100 for each of the two served entities (so the total sums t...

  • 545 Views
  • 1 replies
  • 1 kudos
Latest Reply
emma_s
Databricks Employee
  • 1 kudos

Hi,   Just been looking into this for you. All the docs I found suggested it would reject to I tried testing it myself. I tried setting traffic_config with various percentage combinations that don't sum to 100, on both Azure and AWS: Config (A / B) ...

  • 1 kudos
suryaprayaga
by New Contributor III
  • 945 Views
  • 2 replies
  • 3 kudos

Resolved! Snowflake OR Databricks

I am doing a case choice analysis on whether I should have my data in Snowflake to support my Customer workloads or have them migrated to Databricks to do the same.What has been your experience especially in handling large volumes of data especially ...

  • 945 Views
  • 2 replies
  • 3 kudos
Latest Reply
DoTA
Contributor III
  • 3 kudos

Given the pace in the new AI era, which platform have the most inovations that brings value to customers clearly wins it. And we can see Databricks is releasing new features on weekly basis, they are clearly not behind.

  • 3 kudos
1 More Replies
d_szepietowska
by New Contributor II
  • 948 Views
  • 1 replies
  • 0 kudos

Serving endpoint - automatic system updates

Dear Community I need some support with investigation related to Serving Endpoints. Recently some of endpoints with deployed ML models display message:This endpoint is out of compliance because it is too old and automatic system updates have failed.P...

  • 948 Views
  • 1 replies
  • 0 kudos
Latest Reply
MoJaMa
Databricks Employee
  • 0 kudos

Hi, Could you provide a workspaceID and endpointID and any other details such as screenshots? Will check to see what I can find based on those details, ~Mo.

  • 0 kudos
LPBigot
by New Contributor
  • 396 Views
  • 1 replies
  • 0 kudos

Can't select Serverless GPU on notebooks

When trying to use serverless gpu on a notebook, I get this error. Serverless CPU works fine though.Can you help me understand why I get this?

databricks_bug.png
  • 396 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ashwin_DSA
Databricks Employee
  • 0 kudos

Hi @LPBigot, Since Serverless CPU is working but Serverless GPU is failing when the notebook tries to attach compute, could you confirm a couple of things so we can narrow this down? Is this a paid/Premium+ workspace, or is it a free/trial-style setu...

  • 0 kudos
pjvi
by New Contributor II
  • 537 Views
  • 1 replies
  • 0 kudos

Free Edition CrossValidator not working because of internal caching

Hi all,I am getting the following error when using CrossValidator in Databricks Free Edition v5 (see attachment):In shared or serverless cluster, SPARK_ML_TMP_DFS_PATH environmental variable must be set to a UC volume path like '/Volumes/...' in orde...

  • 537 Views
  • 1 replies
  • 0 kudos
Latest Reply
balajij8
Esteemed Contributor II
  • 0 kudos

What you are encountering is an expected behavior on Databricks serverless. Because CrossValidator (along with other Spark ML tuning estimators) relies heavily on internal DataFrame caching to optimize iterative model training, Databricks enforces st...

  • 0 kudos
sergio-calderon
by New Contributor II
  • 3460 Views
  • 4 replies
  • 2 kudos

How to use Databricks secrets on MLFlow conda dependencies?

Hi!Do you know if it's correct to use the plain user and token for installing a custom dependency (an internal python package) in a mlflow registered model? (it's the only way I get it working because if not it can't install the dependency) It works,...

  • 3460 Views
  • 4 replies
  • 2 kudos
Latest Reply
iyashk-DB
Databricks Employee
  • 2 kudos

The dynamic conda.yaml generation trick doesn't actually fix this, and you already found that out yourself. Whatever ends up in the final logged conda.yaml is what gets read at environment build time, so if a plaintext credential has to be in that fi...

  • 2 kudos
3 More Replies
nepiskopos
by New Contributor III
  • 902 Views
  • 2 replies
  • 0 kudos

AutoML on Azure Databricks as of June 2026

Hello everybody,My team and I are facing a sudden and unexpected issue with training Forecasting models utilizing AutoML on Azure Databricks. As of two weeks ago, we privatized our Azure Databricks environment behind a private network, accessible onl...

  • 902 Views
  • 2 replies
  • 0 kudos
Latest Reply
iyashk-DB
Databricks Employee
  • 0 kudos

Databricks AutoML is still around and actively used as of mid-2026, so nothing's wrong on that front, but there's a nuance worth knowing: starting with Databricks Runtime 18.0 ML, AutoML was removed as a built-in library on classic compute. It's stil...

  • 0 kudos
1 More Replies
ShaRKS
by New Contributor
  • 379 Views
  • 1 replies
  • 0 kudos

Notebooks for Labs

Hello can someone tell me where do I find the notebooks in the demo sessions of Associate ML pathways? 

  • 379 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ashwin_DSA
Databricks Employee
  • 0 kudos

Hi @ShaRKS, If you are taking the Associate ML pathway, the notebooks are typically available through the course's lab environment rather than as separate downloadable assets. Databricks course pages for the ML curriculum note that lecture notebooks ...

  • 0 kudos
Labels