- 1135 Views
- 6 replies
- 2 kudos
Resolved! What is the most efficient way of running sentence-transformers on a Spark DataFrame column?
We're trying to run the bundled sentence-transformers library from SBert in a notebook running Databricks ML 16.4 on an AWS g4dn.2xlarge [T4] instance.However, we're experiencing out of memory crashes and are wondering what the optimal to run sentenc...
- 1135 Views
- 6 replies
- 2 kudos
- 1420 Views
- 8 replies
- 0 kudos
Resolved! Importing sentence-transformers no longer works on Databricks runtime 17.2 ML
In Databricks Runtime 16.4 LTS for Machine Learning, I am used to be able to import sentence-transformers without any installation as it is part of the runtime with from sentence_transformers import SentenceTransformer.In this case I am running on a ...
- 1420 Views
- 8 replies
- 0 kudos
- 0 kudos
I now upgraded to the new 17.3 LTS ML and it now works. I didn't try 17.2 ML, but with 17.3 ML available, I don't see any reason to use it anymore.
- 0 kudos
- 4908 Views
- 6 replies
- 1 kudos
Resolved! Model Serving Endpoint Creation through API
Hello,I am trying to create a model serving endpoint via the API as explained here: https://docs.databricks.com/api/workspace/servingendpoints/createI created a trusted IAM role with access to DynamoDB for the feature store. I try to use this field,"...
- 4908 Views
- 6 replies
- 1 kudos
- 1 kudos
If you're using the databricks terraform provider, make sure the role's name matches the instance-profile name.If not, use the `iam_role_arn` attribute to explicitly set the role's arn when creating the databricks instance profileresource "databricks...
- 1 kudos
- 3634 Views
- 1 replies
- 0 kudos
AutoML "need to sample" not working as expected
tl; dr:When the AutoML run realizes it needs to do sampling because the driver / worker node memory is not enough to load / process the entire dataset, it fails. A sample weight column is NOT provided by me, but I believe somewhere in the process the...
- 3634 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @sangramraje , sorry for the late response. I wanted to check in to see if this is still an issue with the latest release? Please let me know. Cheers, Louis.
- 0 kudos
- 3742 Views
- 1 replies
- 1 kudos
AutoML Doesn't Work Due to Not being able to generate the EDA notebook
HiI'm trying run AutoML classification experiment with a dataset that I have made, and am experiencing this issue even after I have purposely downsampled my dataset before running it into the AutoML experiment. It appears that there is no way for me ...
- 3742 Views
- 1 replies
- 1 kudos
- 1 kudos
Hey @adoodsonruby , sorry this got lost in the shuffle. Have you tried again recently? I believe limits have been increased that would remove this impediment. Let us know, Louis.
- 1 kudos
- 3415 Views
- 1 replies
- 0 kudos
Error to create an endpoint of databricks with 2 primary keys online table
I have a delta table that has a primary key conformed by 2 fields (accountId,ruleModelVersionDesc) and I have also created an online table that has the same primary key, but when I create a feature spec to create an endpoint I get the following error...
- 3415 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @lchicoma , sorry for the delayed response. Thanks for sharing the error and context—this looks like a parsing issue in the feature specification rather than a problem with Delta or the runtime versions. What changed recently There was an inci...
- 0 kudos
- 965 Views
- 1 replies
- 0 kudos
🐞 Stuck on LightGBM Distributed Training in PySpark – Hanging After Socket Communication
My Setup:I'm trying to run distributed LightGBM training using synapseml.lightgbm.LightGBMRegressor in PySpark.Cluster Details:Spark version: 3.5.1 (compatible with PySpark 3.5.6)PySpark version: 3.5.6synapseml: v0.11.1 (latest)Spark Cluster: 3 Hetzn...
- 965 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @amanjethani , Thanks for laying out the setup and symptoms so clearly. The hang likely occurs because LightGBM’s distributed network either doesn’t fully form between executors or because the expected task count doesn’t match actual tasks, leadin...
- 0 kudos
- 3413 Views
- 1 replies
- 0 kudos
Can't query Legacy Serving Endpoint
Hi,I was able to deploy an endpoint using legacy serving (It's the only option we have to deploy endpoints in DB). Now I am having trouble querying the endpoint itself. When I try to query it I get the following error: Here is the code I am using ...
- 3413 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @semsim , sorry for the delayed response. Thanks for the screenshot—this pinpoints the problem. Root cause from the error Your model’s predict path is trying to create or write to /Workspace/Shared, and the serving container does not permit t...
- 0 kudos
- 4020 Views
- 1 replies
- 1 kudos
Multi-tenant recommendation system (Machine learning)
Hello,I am looking to build a multi-tenant machine learning recommender system in Azure Databricks. The idea is to have a single shared model, where each tenant can use the same model to train on their own unique dataset. Essentially, while the model...
- 4020 Views
- 1 replies
- 1 kudos
- 1 kudos
@Kasen , sorry for the delayed response. Here are some things to consider regarding your question. Azure Databricks is well-suited for a shared-architecture, tenant‑isolated recommender system. Below is a pragmatic blueprint, the isolation model o...
- 1 kudos
- 3530 Views
- 1 replies
- 0 kudos
Determine exact location of MLflow model tracking and model registry files and the Backend Stores
I would like to determine the exact location of:1. MLflow model tracking files2. Model registry files (with Workspace Model Registry)as according to the documentation it is mentioned that: "All methods copy the model into a secure location managed by...
- 3530 Views
- 1 replies
- 0 kudos
- 0 kudos
Greetings @ScyLukb , You’re right that the docs say the Workspace Model Registry copies models to a “secure location” but don’t name it prominently. Here’s where those files actually live and how to discover the configured stores. Locations of MLflo...
- 0 kudos
- 3290 Views
- 1 replies
- 0 kudos
Hyperopt (15.4 LTS ML) ignores autologger settings
I use ML Flow Experiment to store models once they leave very early tests and development. I switched lately to 15.4 LTS ML and was hit by unhinged Hyperopt behavior:it was creating Experiment logs ignoring i) autologger is off on the workspace level...
- 3290 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @art1 , sorry this post got lost in the shuffle. Here are some things to consider regarding your question: Thanks for flagging this—what you’re seeing is expected given how Databricks integrates Hyperopt with MLflow, and there are clear ways t...
- 0 kudos
- 3356 Views
- 1 replies
- 0 kudos
Working with pyspark dataframe with machine learning libraries / statistical model libraries
Hi Team, I am working with huge volume of data (50GB) and i decompose the time series data using the statsmodel.Having said that the major challenge i am facing is the compatibility of the pyspark dataframe with the machine learning algorithms. altho...
- 3356 Views
- 1 replies
- 0 kudos
- 0 kudos
Greetings @javeed , You’re right to call out the friction between a PySpark DataFrame and many Python ML libraries like statsmodels; most Python ML stacks expect pandas, while Spark is distributed-first. Here’s how to bridge that gap efficiently fo...
- 0 kudos
- 3374 Views
- 1 replies
- 0 kudos
databricks-vectorsearch 0.53 unable to use similarity_search()
I have an issue with databricks-vectorsearch package. Version 0.51 suddenly stopped working this week because:It now expected me to provide azure_tenant_id in addition to service principal's client ID and secret.After supplying tenant ID, it showed s...
- 3374 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @snaveedgm , This is interesting - can you double-check that the service principal has CAN QUERY on the embedding endpoint used for ingestion and/or querying (databricks-bge-large-en in your case)? Even though your direct REST test works, double-c...
- 0 kudos
- 3331 Views
- 1 replies
- 0 kudos
ML Solution for unstructured data containing Images and videos
Hi,I have a use case of developing an entire ML solution within Databricks starting from ingestion to inference and monitoring, but the problem is that we have unstructured data containing Images and Video for training the model using frameworks such...
- 3331 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @aswinkks , This is a very broad question, but generally, when dealing with video data, you convert the videos to images and have a system in place for training and another for inference. This Databricks blog posts explains how to set up a video ...
- 0 kudos
- 12674 Views
- 4 replies
- 3 kudos
Resolved! How to PREVENT mlflow's autologging from logging ALL runs?
I am logging runs from jupyter notebook. the cells which has `mlflow.sklearn.autlog()` behaves as expected. but, the cells which has .fit() method being called on sklearn's estimators are also being logged as runs without explicitly mentioning `mlflo...
- 12674 Views
- 4 replies
- 3 kudos
- 3 kudos
It looks like MLflow auto-logging is kicking in by default whenever you call .fit(), which is why you’re seeing runs even without explicitly using mlflow.sklearn.autolog(). To fix this, you can disable the global autologging and only trigger it when ...
- 3 kudos
