Most of my time-series work hits the same wall. The models only learn from scenarios that already happened.Rare events barely show up in the training data. Demand spikes, stockouts, and extreme regimes are exactly the cases I care about. They are als...
Hi everybody,I have a scenario where we have multiple teams working with Python and R, and this teams uses a lot of different libraries. Because of this dozen of libraries, the cluster start took much time. Then I created a Docker image, where I can ...
Mojang's commitment to regularly releasing a new Minecraft Patch Download demonstrates their deep respect and genuine appreciation for the passionate global community that has made Minecraft the most beloved game in history.Click Here
What is Image Annotation?What are the Steps of Image Annotation?What are the Different Techniques of Image Annotation?Types Used in Image AnnotationHow are Companies Handling Image Annotation?Features to Look for in Image Annotation Service Providers...
Great overview of image annotation! The sections on different techniques, real-world use cases, and pricing factors are especially helpful for anyone getting started with AI/ML. Looking forward to exploring the future trends in image annotation!
Hi everyone,I am trying to understand the exact differences between using Workspace experiments versus Unity Catalog experiments, specifically in the context of MLflow autologging (mlflow.autolog()).Does autologging behave differently depending on wh...
Great question! I’d also be interested to know if autologging has any behavioral or permission differences with Unity Catalog experiments, especially around governance and model lineage.
As the AI revolution takes off in 2025, there is a renewed emphasis on adopting a Data-First approach. Organizations are increasingly recognizing the need to establish a robust data foundation while preparing a skilled fleet of Data Engineers to tack...
Absolutely agree! AI is only as powerful as the data behind it. The evolving role of data engineers in building that foundation is definitely worth discussing.
Most time-series forecasting jobs do not come down to one model. Teams run N-BEATS, NHITS, LSTM, and GRU, then struggle to combine them well.I want to open a discussion on model synthesis. Not data cleaning. Not generative AI. I mean merging several ...
Hi,Good topic. A few things that have held up for us.On weighting across horizons, fit weights per horizon step rather than one global set, and derive them from rolling origin backtests so the errors are genuinely out of sample. Models tend to trade ...
Has anyone else had a problem today (2026-09-04) with imports from fastai?The error message points to a problem with the underlying fastcore package:AttributeError: 'Function' object attribute '__doc__' is read-onlyI'm getting the same error using fa...
I’m seeing the same issue today—seems like the recent compute runtime update may be related. Hopefully someone from the fastai/Databricks side can confirm.
When it comes to machine learning, the platform plays a pivotal role in successful implementation. Databricks offers a best-in-class machine learning platform with cutting-edge features such as MLflow, Model Registry, Feature Store, and MLOps, which ...
Great share! Databricks really has built something special on the ML side — MLflow alone has become almost a default for experiment tracking, and pairing it with Model Registry and Feature Store makes the whole path from experimentation to productio...
I am following the guide for tuning a model with Optuna and MlflowSparkStudy. My compute is configured with autoscaling enabled, with 1–2 Spark workers, each with 8 cores and 32 GB of memory. I set n_jobs=2 and trials=100, in mlflow_study.optimize()....
Great questions—especially the distinction between Optuna’s trial-level parallelism and LightGBM’s intra-trial threading. The interaction with Spark autoscaling and data locality is also something I’d love to see documented with a concrete example. C...
Building an End-to-End Streaming NLP Pipeline with GDELT, Azure Data Factory, ADLS Gen2 and DatabricksI recently worked on an end-to-end streaming NLP project using GDELT news data, Azure Data Factory, ADLS Gen2 and Azure Databricks.The goal was not ...
Great end-to-end project! To take it to the next level, consider exploring these native Databricks capabilities.Lakehouse Monitoring: Track data drift and automate model retraining when performance drops.Model Serving: Deploy your models behind serve...
I have been working on a project to understand Databricks end to end, rather than just loading some data and training a model.I picked GDELT news data and the use case is to identify supply chain disruption related news and eventually predict which e...
@kartheek_rao, you are on the right track.Since you are doing clustering of new articles, it's unsupervised learning; you need to understand the feature engineering part more and the EDA part with MLFlow experiments. You have model monitoring as well...
We can't deploy models to Model Serving. Failures started around 2026-08-16 and were intermittent at first — our current production config deployed cleanly on 08-20 — but since then every attempt fails at the feature store lookup setup step, with no ...
Update — reproduced at minimum complexity, and the failure is unobservableWe stopped theorising and bisected with a deliberately trivial model: SimpleImputer + LogisticRegression, logged with fe.log_model, no custom code, on an online store created t...
VS status remains "syncing" indefinitely — rechecked after 12 hours, status still "syncing". The Databricks job completes successfully and the new data is queryable:
@airbotsThe underlying issue is generally a metadata reconciliation lag where the Vector Search index status remains stuck displaying Syncing in Catalog Explorer even after the sync pipeline has completed. The metadata status reporting mechanism gene...
Hi everyone,I'm working on a personal project and would appreciate some advice from people who have experience with machine learning on Databricks.Imagine having a dataset containing thousands of game compatibility records collected from different em...
Yeah, I’d start simple with XGBoost or LightGBM and use AutoML for a quick baseline. Then just focus on the features that actually improve the results.
Hi everyone,I have an image-based deep learning workload running on Azure Databricks, while the training dataset must remain in AWS S3 due to some constraints. We cannot move or replicate the dataset to Azure.Our current architecture is roughly:AWS S...
I’d first check whether the bottleneck is S3/network latency or image decoding. For multi-epoch training, local caching + larger MDS shards might help, but the cross-cloud setup could still be the main issue.
