cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

jitenjha11
by • Databricks Partner
  • 2439 Views
  • 2 replies
  • 0 kudos

Getting error when running databricks deploy bundle command

HI all,I am trying to implement MLOps project using https://github.com/databricks/mlops-stacks repo.I have created azure databricks with Premium (+ Role-based access controls) (Click to change) and following bundle creation and deploy using uRL: http...

  • 2439 Views
  • 2 replies
  • 0 kudos
Latest Reply
Louis_Frolio
Databricks Employee
  • 0 kudos

Greetings @jitenjha11, I did some digging and here is what I found. @emma_s is on the right track. Make sure the identity running the deploy has workspace access and cluster creation rights, and that the workspace is attached to a Unity Catalog metas...

  • 0 kudos
1 More Replies
holyone2
by • New Contributor II
  • 252 Views
  • 2 replies
  • 0 kudos

MLflow traces accepted by StartTraceV3 (200 OK) but never stored: GetTrace returns NOT_FOUND

Hi all,We're seeing cases where traces go missing in an experiment-backed MLflow trace store (not Unity Catalog). The client gets a successful response for both trace writes, and the server echoes back the same trace id, but the trace is never stored...

  • 252 Views
  • 2 replies
  • 0 kudos
Latest Reply
anuj_lathi
Databricks Employee
  • 0 kudos

Short version: your new observation rules out a client-side drop. If a retried StartTraceV3 with the same id returns "already exists" while GetTrace and SearchTraces return NOT_FOUND, the server accepted and recorded something for that id, but the r...

  • 0 kudos
1 More Replies
FrankAzzollini
by • New Contributor II
  • 187 Views
  • 0 replies
  • 1 kudos

Document AI is a pipeline, not a model

When a financial document needs to be approved or rejected, there usually isn't one model making the decision.In the check / payment-slip systems I worked on, the pipeline looked more like this:1. Multimodal models + OCR to extract fields2. YOLO to d...

  • 187 Views
  • 0 replies
  • 1 kudos
ThiamLee
by • Contributor
  • 524 Views
  • 3 replies
  • 1 kudos

Resolved! How do you train forecasting models for events that never happened?

Most of my time-series work hits the same wall. The models only learn from scenarios that already happened.Rare events barely show up in the training data. Demand spikes, stockouts, and extreme regimes are exactly the cases I care about. They are als...

  • 524 Views
  • 3 replies
  • 1 kudos
Latest Reply
niteshm
Contributor
  • 1 kudos

Hey @ThiamLee , great question, and one I've wrestled with too. Haven't tried Remix Labs specifically, tho.My take, after going back and forth between pure historical data and augmentation: real data first, augment second, but only where augmentation...

  • 1 kudos
2 More Replies
fsimoes
by • New Contributor II
  • 5384 Views
  • 3 replies
  • 1 kudos

Resolved! Docker image with libraries + MLFlow Experiments

Hi everybody,I have a scenario where we have multiple teams working with Python and R, and this teams uses a lot of different libraries. Because of this dozen of libraries, the cluster start took much time. Then I created a Docker image, where I can ...

  • 5384 Views
  • 3 replies
  • 1 kudos
Latest Reply
jackleo
New Contributor II
  • 1 kudos

Mojang's commitment to regularly releasing a new Minecraft Patch Download  demonstrates their deep respect and genuine appreciation for the passionate global community that has made Minecraft the most beloved game in history.Click Here

  • 1 kudos
2 More Replies
ben_ai
by • New Contributor II
  • 333 Views
  • 2 replies
  • 2 kudos

Image Annotation

What is Image Annotation?What are the Steps of Image Annotation?What are the Different Techniques of Image Annotation?Types Used in Image AnnotationHow are Companies Handling Image Annotation?Features to Look for in Image Annotation Service Providers...

  • 333 Views
  • 2 replies
  • 2 kudos
Latest Reply
ThiamLee
Contributor
  • 2 kudos

Great overview of image annotation! The sections on different techniques, real-world use cases, and pricing factors are especially helpful for anyone getting started with AI/ML. Looking forward to exploring the future trends in image annotation! 

  • 2 kudos
1 More Replies
kunduruanil
by • New Contributor III
  • 1045 Views
  • 5 replies
  • 6 kudos

Resolved! Difference between Workspace and Unity Catalog experiments when using MLflow autologging?

Hi everyone,I am trying to understand the exact differences between using Workspace experiments versus Unity Catalog experiments, specifically in the context of MLflow autologging (mlflow.autolog()).Does autologging behave differently depending on wh...

  • 1045 Views
  • 5 replies
  • 6 kudos
Latest Reply
ThiamLee
Contributor
  • 6 kudos

Great question! I’d also be interested to know if autologging has any behavioral or permission differences with Unity Catalog experiments, especially around governance and model lineage.

  • 6 kudos
4 More Replies
infinitylearnin
by • New Contributor III
  • 1159 Views
  • 2 replies
  • 1 kudos

Data practitioner in AI Era

As the AI revolution takes off in 2025, there is a renewed emphasis on adopting a Data-First approach. Organizations are increasingly recognizing the need to establish a robust data foundation while preparing a skilled fleet of Data Engineers to tack...

  • 1159 Views
  • 2 replies
  • 1 kudos
Latest Reply
ThiamLee
Contributor
  • 1 kudos

Absolutely agree! AI is only as powerful as the data behind it. The evolving role of data engineers in building that foundation is definitely worth discussing.

  • 1 kudos
1 More Replies
ThiamLee
by • Contributor
  • 508 Views
  • 1 replies
  • 1 kudos

Resolved! How are you combining forecasting models for time-series work?

Most time-series forecasting jobs do not come down to one model. Teams run N-BEATS, NHITS, LSTM, and GRU, then struggle to combine them well.I want to open a discussion on model synthesis. Not data cleaning. Not generative AI. I mean merging several ...

  • 508 Views
  • 1 replies
  • 1 kudos
Latest Reply
Islam_hoti
Contributor
  • 1 kudos

Hi,Good topic. A few things that have held up for us.On weighting across horizons, fit weights per horizon step rather than one global set, and derive them from rolling origin backtests so the errors are genuinely out of sample. Models tend to trade ...

  • 1 kudos
barnabywalker
by • New Contributor III
  • 1126 Views
  • 8 replies
  • 4 kudos

Resolved! fastai import in databricks broken

Has anyone else had a problem today (2026-09-04) with imports from fastai?The error message points to a problem with the underlying fastcore package:AttributeError: 'Function' object attribute '__doc__' is read-onlyI'm getting the same error using fa...

  • 1126 Views
  • 8 replies
  • 4 kudos
Latest Reply
ThiamLee
Contributor
  • 4 kudos

I’m seeing the same issue today—seems like the recent compute runtime update may be related. Hopefully someone from the fastai/Databricks side can confirm.

  • 4 kudos
7 More Replies
Akash_Wadhankar
by • Databricks Partner
  • 1929 Views
  • 2 replies
  • 1 kudos

Learn Databricks AI medium article series for fellow learners.

When it comes to machine learning, the platform plays a pivotal role in successful implementation. Databricks offers a best-in-class machine learning platform with cutting-edge features such as MLflow, Model Registry, Feature Store, and MLOps, which ...

Machine Learning
DatabricksML MachineLearning AI FeatureStore DecisionScience
  • 1929 Views
  • 2 replies
  • 1 kudos
Latest Reply
ThiamLee
Contributor
  • 1 kudos

Great share! Databricks really has built something special on the ML side — MLflow alone has become almost a default for experiment tracking, and pairing it with Model Registry and Feature Store makes the whole path from experimentation to productio...

  • 1 kudos
1 More Replies
AdamIH123
by • New Contributor III
  • 423 Views
  • 2 replies
  • 1 kudos

Tuning with Optuna and MlflowSparkStudy

I am following the guide for tuning a model with Optuna and MlflowSparkStudy. My compute is configured with autoscaling enabled, with 1–2 Spark workers, each with 8 cores and 32 GB of memory. I set n_jobs=2 and trials=100, in mlflow_study.optimize()....

Machine Learning
mlflow
mlflow_study
mlflow_study.optimize
MlflowSparkStudy
optuna
  • 423 Views
  • 2 replies
  • 1 kudos
Latest Reply
ThiamLee
Contributor
  • 1 kudos

Great questions—especially the distinction between Optuna’s trial-level parallelism and LightGBM’s intra-trial threading. The interaction with Spark autoscaling and data locality is also something I’d love to see documented with a concrete example. C...

  • 1 kudos
1 More Replies
kartheek_rao
by • New Contributor III
  • 501 Views
  • 1 replies
  • 4 kudos

End-to-End Streaming NLP Pipeline with GDELT, Azure Data Factory, ADLS Gen2 and Databricks

Building an End-to-End Streaming NLP Pipeline with GDELT, Azure Data Factory, ADLS Gen2 and DatabricksI recently worked on an end-to-end streaming NLP project using GDELT news data, Azure Data Factory, ADLS Gen2 and Azure Databricks.The goal was not ...

  • 501 Views
  • 1 replies
  • 4 kudos
Latest Reply
kunduruanil
New Contributor III
  • 4 kudos

Great end-to-end project! To take it to the next level, consider exploring these native Databricks capabilities.Lakehouse Monitoring: Track data drift and automate model retraining when performance drops.Model Serving: Deploy your models behind serve...

  • 4 kudos
kartheek_rao
by • New Contributor III
  • 795 Views
  • 4 replies
  • 3 kudos

End-to-End Streaming NLP Pipeline with GDELT, Azure Data Factory, ADLS Gen2 and Databricks

I have been working on a project to understand Databricks end to end, rather than just loading some data and training a model.I picked GDELT news data and the use case is to identify supply chain disruption related news and eventually predict which e...

  • 795 Views
  • 4 replies
  • 3 kudos
Latest Reply
kunduruanil
New Contributor III
  • 3 kudos

@kartheek_rao, you are on the right track.Since you are doing clustering of new articles, it's unsupervised learning; you need to understand the feature engineering part more and the EDA part with MLFlow experiments. You have model monitoring as well...

  • 3 kudos
3 More Replies
ticusss
by • New Contributor II
  • 626 Views
  • 2 replies
  • 1 kudos

Model Serving An internal error occurred during feature store lookup all deploys failing

We can't deploy models to Model Serving. Failures started around 2026-08-16 and were intermittent at first — our current production config deployed cleanly on 08-20 — but since then every attempt fails at the feature store lookup setup step, with no ...

  • 626 Views
  • 2 replies
  • 1 kudos
Latest Reply
ticusss
New Contributor II
  • 1 kudos

Update — reproduced at minimum complexity, and the failure is unobservableWe stopped theorising and bisected with a deliberately trivial model: SimpleImputer + LogisticRegression, logged with fe.log_model, no custom code, on an online store created t...

  • 1 kudos
1 More Replies
Labels