Databricks Community

Hi @ScyLukb , This is a common and frustrating problem that occurs when the Model Serving environment's built-in dependencies conflict with your model's specific requirements. The root cause is that the Model Serving environment tries to install its ...

Hi @Mario_D , Thanks for bringing this to our attention, I will pass this information along to the appropriate team!

You’re running into a fundamental limitation: score_batch does point‑in‑time feature lookups and batch scoring, but it doesn’t support recursive multi‑step forecasting where predictions update features for subsequent timesteps. Feature Store looks up...

hi @tooooods , This is a classic challenge in distributed computing, and your observation is spot on. The ModuleNotFoundError on the workers, despite the UI and API showing the library as "Installed," is the key symptom. This happens because TorchDis...

Greetings @hawa ,  Thanks for sharing the details—this looks like a combination of registration and configuration issues that commonly surface with the MLflow LangChain flavor on Databricks. What’s going wrong The registered model name should be a fu...

You are encountering issues serving a SynapseML model (with Maven dependencies) via Databricks Model Serving Endpoints, and the deployment works fine on general-purpose clusters but fails for the serving endpoint. This is a well-known issue with Data...

Both the parent and child runs of a Hyperopt sweep in Databricks are, by default, influenced by the experiment associated with the notebook context rather than the explicit experiment passed to mlflow.start_run(). As you noticed, child runs remain in...

The main reason your Hugging Face embedding model endpoint is taking much longer than Databricks’ own large_bge_en model to build a vector search index is likely due to differences in operational architecture and performance optimizations between ext...

The error you’re seeing—SPARK-5063 CONTEXT_ONLY_VALID_ON_DRIVER—arises when trying to serialize or use objects (such as functions) defined in Databricks notebooks from workers rather than the driver. This issue is especially common with Python functi...

Hey @steve2 ,  short answer: these look like TensorBoard event files, likely created by a library that briefly initialized a TensorBoard logger or writer during one of your training/serving runs; the sparse_logs folder naming and “manager stage: Mode...

Greetings @VELU1122 ,  you’re correct that the Databricks Model Serving container is isolated, so you can’t rely on cluster-only affordances like mounts or executor-distributed file utilities. The reliable way to read from Unity Catalog (UC) Volumes ...

Hello @grajee ,  I can see you're dealing with two separate issues here. Let me address both: Issue 1: The model_id column (request_metadata MAP type) You're correct that request_metadata is a MAP type and can't be directly used as the model_id colum...

Greetings @sharpbetty  Great question! Databricks AutoML's "glass box" approach actually gives you several options to customize preprocessing beyond the default StandardScaler. Here are two practical approaches: Option A: Pre-process Features Before ...