cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

barnabywalker
by New Contributor
  • 309 Views
  • 8 replies
  • 4 kudos

Resolved! fastai import in databricks broken

Has anyone else had a problem today (2026-09-04) with imports from fastai?The error message points to a problem with the underlying fastcore package:AttributeError: 'Function' object attribute '__doc__' is read-onlyI'm getting the same error using fa...

  • 309 Views
  • 8 replies
  • 4 kudos
Latest Reply
ThiamLee
New Contributor III
  • 4 kudos

I’m seeing the same issue today—seems like the recent compute runtime update may be related. Hopefully someone from the fastai/Databricks side can confirm.

  • 4 kudos
7 More Replies
Akash_Wadhankar
by Databricks Partner
  • 1689 Views
  • 2 replies
  • 1 kudos

Learn Databricks AI medium article series for fellow learners.

When it comes to machine learning, the platform plays a pivotal role in successful implementation. Databricks offers a best-in-class machine learning platform with cutting-edge features such as MLflow, Model Registry, Feature Store, and MLOps, which ...

Machine Learning
DatabricksML MachineLearning AI FeatureStore DecisionScience
  • 1689 Views
  • 2 replies
  • 1 kudos
Latest Reply
ThiamLee
New Contributor III
  • 1 kudos

Great share! Databricks really has built something special on the ML side — MLflow alone has become almost a default for experiment tracking, and pairing it with Model Registry and Feature Store makes the whole path from experimentation to productio...

  • 1 kudos
1 More Replies
AdamIH123
by New Contributor III
  • 91 Views
  • 2 replies
  • 1 kudos

Tuning with Optuna and MlflowSparkStudy

I am following the guide for tuning a model with Optuna and MlflowSparkStudy. My compute is configured with autoscaling enabled, with 1–2 Spark workers, each with 8 cores and 32 GB of memory. I set n_jobs=2 and trials=100, in mlflow_study.optimize()....

Machine Learning
mlflow
mlflow_study
mlflow_study.optimize
MlflowSparkStudy
optuna
  • 91 Views
  • 2 replies
  • 1 kudos
Latest Reply
ThiamLee
New Contributor III
  • 1 kudos

Great questions—especially the distinction between Optuna’s trial-level parallelism and LightGBM’s intra-trial threading. The interaction with Spark autoscaling and data locality is also something I’d love to see documented with a concrete example. C...

  • 1 kudos
1 More Replies
infinitylearnin
by New Contributor III
  • 913 Views
  • 1 replies
  • 0 kudos

Data practitioner in AI Era

As the AI revolution takes off in 2025, there is a renewed emphasis on adopting a Data-First approach. Organizations are increasingly recognizing the need to establish a robust data foundation while preparing a skilled fleet of Data Engineers to tack...

  • 913 Views
  • 1 replies
  • 0 kudos
Latest Reply
kunduruanil
New Contributor II
  • 0 kudos

@infinitylearnin, the above podcast link was changed; it's not working !! 

  • 0 kudos
kartheek_rao
by New Contributor III
  • 275 Views
  • 1 replies
  • 3 kudos

End-to-End Streaming NLP Pipeline with GDELT, Azure Data Factory, ADLS Gen2 and Databricks

Building an End-to-End Streaming NLP Pipeline with GDELT, Azure Data Factory, ADLS Gen2 and DatabricksI recently worked on an end-to-end streaming NLP project using GDELT news data, Azure Data Factory, ADLS Gen2 and Azure Databricks.The goal was not ...

  • 275 Views
  • 1 replies
  • 3 kudos
Latest Reply
kunduruanil
New Contributor II
  • 3 kudos

Great end-to-end project! To take it to the next level, consider exploring these native Databricks capabilities.Lakehouse Monitoring: Track data drift and automate model retraining when performance drops.Model Serving: Deploy your models behind serve...

  • 3 kudos
kunduruanil
by New Contributor II
  • 265 Views
  • 3 replies
  • 3 kudos

Resolved! Difference between Workspace and Unity Catalog experiments when using MLflow autologging?

Hi everyone,I am trying to understand the exact differences between using Workspace experiments versus Unity Catalog experiments, specifically in the context of MLflow autologging (mlflow.autolog()).Does autologging behave differently depending on wh...

  • 265 Views
  • 3 replies
  • 3 kudos
Latest Reply
ivanvyd
New Contributor II
  • 3 kudos

@kunduruanil with MLflow 3.8.1, you control two storage decisions:The tracking URI selects the server that stores experiment and run metadata.The experiment's artifact location selects where MLflow stores model files and other artifacts.mlflow.autolo...

  • 3 kudos
2 More Replies
kartheek_rao
by New Contributor III
  • 458 Views
  • 4 replies
  • 3 kudos

End-to-End Streaming NLP Pipeline with GDELT, Azure Data Factory, ADLS Gen2 and Databricks

I have been working on a project to understand Databricks end to end, rather than just loading some data and training a model.I picked GDELT news data and the use case is to identify supply chain disruption related news and eventually predict which e...

  • 458 Views
  • 4 replies
  • 3 kudos
Latest Reply
kunduruanil
New Contributor II
  • 3 kudos

@kartheek_rao, you are on the right track.Since you are doing clustering of new articles, it's unsupervised learning; you need to understand the feature engineering part more and the EDA part with MLFlow experiments. You have model monitoring as well...

  • 3 kudos
3 More Replies
ben_ai
by New Contributor
  • 119 Views
  • 1 replies
  • 0 kudos

Image Annotation

What is Image Annotation?What are the Steps of Image Annotation?What are the Different Techniques of Image Annotation?Types Used in Image AnnotationHow are Companies Handling Image Annotation?Features to Look for in Image Annotation Service Providers...

  • 119 Views
  • 1 replies
  • 0 kudos
Latest Reply
kunduruanil
New Contributor II
  • 0 kudos

Image annotation is the process of adding labels or tags to images so an AI/ML model can understand. You as a human can see with your eyes what is present in the image for the AI to understand it. It requires training data to prepare training data. I...

  • 0 kudos
ticusss
by New Contributor II
  • 307 Views
  • 2 replies
  • 1 kudos

Model Serving An internal error occurred during feature store lookup all deploys failing

We can't deploy models to Model Serving. Failures started around 2026-08-16 and were intermittent at first — our current production config deployed cleanly on 08-20 — but since then every attempt fails at the feature store lookup setup step, with no ...

  • 307 Views
  • 2 replies
  • 1 kudos
Latest Reply
ticusss
New Contributor II
  • 1 kudos

Update — reproduced at minimum complexity, and the failure is unobservableWe stopped theorising and bisected with a deliberately trivial model: SimpleImputer + LogisticRegression, logged with fe.log_model, no custom code, on an online store created t...

  • 1 kudos
1 More Replies
airbots
by New Contributor II
  • 260 Views
  • 2 replies
  • 1 kudos

Vector Store update stale in Syncing status even the actual sync task is done.

VS status remains "syncing" indefinitely — rechecked after 12 hours, status still "syncing". The Databricks job completes successfully and the new data is queryable:

  • 260 Views
  • 2 replies
  • 1 kudos
Latest Reply
balajij8
Esteemed Contributor II
  • 1 kudos

@airbotsThe underlying issue is generally a metadata reconciliation lag where the Vector Search index status remains stuck displaying Syncing in Catalog Explorer even after the sync pipeline has completed. The metadata status reporting mechanism gene...

  • 1 kudos
1 More Replies
lucas1147
by New Contributor II
  • 470 Views
  • 2 replies
  • 2 kudos

Using Machine Learning to Improve Emulator Compatibility Predictions

Hi everyone,I'm working on a personal project and would appreciate some advice from people who have experience with machine learning on Databricks.Imagine having a dataset containing thousands of game compatibility records collected from different em...

  • 470 Views
  • 2 replies
  • 2 kudos
Latest Reply
ThiamLee
New Contributor III
  • 2 kudos

Yeah, I’d start simple with XGBoost or LightGBM and use AutoML for a quick baseline. Then just focus on the features that actually improve the results.

  • 2 kudos
1 More Replies
aswinkks
by New Contributor III
  • 630 Views
  • 4 replies
  • 7 kudos

Resolved! ML Training low File I/O and Throughout

Hi everyone,I have an image-based deep learning workload running on Azure Databricks, while the training dataset must remain in AWS S3 due to some constraints. We cannot move or replicate the dataset to Azure.Our current architecture is roughly:AWS S...

  • 630 Views
  • 4 replies
  • 7 kudos
Latest Reply
ThiamLee
New Contributor III
  • 7 kudos

I’d first check whether the bottleneck is S3/network latency or image decoding. For multi-epoch training, local caching + larger MDS shards might help, but the cross-cloud setup could still be the main issue.

  • 7 kudos
3 More Replies
MageshS
by New Contributor II
  • 663 Views
  • 1 replies
  • 1 kudos

Resolved! Traffic split behavior when traffic_percentage values across served entities sum to more than 100%

Hi all,I'm configuring a Model Serving endpoint with two served entities and ran into some unexpected behavior while testing different traffic_config splits.When I set traffic_percentage to 100 for each of the two served entities (so the total sums t...

  • 663 Views
  • 1 replies
  • 1 kudos
Latest Reply
emma_s
Databricks Employee
  • 1 kudos

Hi,   Just been looking into this for you. All the docs I found suggested it would reject to I tried testing it myself. I tried setting traffic_config with various percentage combinations that don't sum to 100, on both Azure and AWS: Config (A / B) ...

  • 1 kudos
suryaprayaga
by Contributor
  • 1305 Views
  • 2 replies
  • 3 kudos

Resolved! Snowflake OR Databricks

I am doing a case choice analysis on whether I should have my data in Snowflake to support my Customer workloads or have them migrated to Databricks to do the same.What has been your experience especially in handling large volumes of data especially ...

  • 1305 Views
  • 2 replies
  • 3 kudos
Latest Reply
DoTA
Valued Contributor
  • 3 kudos

Given the pace in the new AI era, which platform have the most inovations that brings value to customers clearly wins it. And we can see Databricks is releasing new features on weekly basis, they are clearly not behind.

  • 3 kudos
1 More Replies
d_szepietowska
by New Contributor II
  • 970 Views
  • 1 replies
  • 0 kudos

Serving endpoint - automatic system updates

Dear Community I need some support with investigation related to Serving Endpoints. Recently some of endpoints with deployed ML models display message:This endpoint is out of compliance because it is too old and automatic system updates have failed.P...

  • 970 Views
  • 1 replies
  • 0 kudos
Latest Reply
MoJaMa
Databricks Employee
  • 0 kudos

Hi, Could you provide a workspaceID and endpointID and any other details such as screenshots? Will check to see what I can find based on those details, ~Mo.

  • 0 kudos
Labels