cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

miahopman
by New Contributor II
  • 3077 Views
  • 1 replies
  • 1 kudos

AutoML Trials Failing

Sometimes an AutoML experiment will have all trials fail and I cannot figure out what is causing it. Each individual run reports a validation f1 value but the source notebook is not available so I cannot track down the error. This seems to happen at ...

  • 3077 Views
  • 1 replies
  • 1 kudos
145093
by New Contributor II
  • 5981 Views
  • 2 replies
  • 2 kudos

MLFlow model loading taking long time and "model serving" failing during init

I am trying to load a simple Minmaxscaler model that was logged as a run through spark's ML Pipeline api for reuse. On average it takes 40+ seconds just to load the model with the following example: This is fine and the model transforms my data corre...

simple model load sometimes the model takes almost 3 min just to load
  • 5981 Views
  • 2 replies
  • 2 kudos
Latest Reply
DanSimpson
New Contributor II
  • 2 kudos

Hello,Any solutions found for this issue?I'm serving up a large number of models at a time, but since we converted to PySpark (due to our data demands), the mlflow.spark.load_model() is taking hours.Part of the reason to switch to spark was to help w...

  • 2 kudos
1 More Replies
Orion
by New Contributor
  • 1345 Views
  • 1 replies
  • 0 kudos
  • 1345 Views
  • 1 replies
  • 0 kudos
Latest Reply
oliverjack
New Contributor II
  • 0 kudos

Large Language Models (LLMs) revolutionize the insurance sector, automating support and enhancing accuracy across claims and underwriting. They're crucial for market analysis processing extensive textual data, and we welcome your insights on deployin...

  • 0 kudos
camazg
by New Contributor
  • 1444 Views
  • 0 replies
  • 0 kudos

Terraform - Creating Jobs

Hello, Does anyone know how to create a job via Terraform, that automatically overwrites/updates an existing job with the same name?Tried a few different methods but there doesn't seem to be a clean approach. Wondering if anyone has worked this out?I...

  • 1444 Views
  • 0 replies
  • 0 kudos
Anon
by New Contributor
  • 1590 Views
  • 0 replies
  • 0 kudos

databricks.mrm import ModelRiskApi

Hi, I am trying to import/access this module by DBX. However I have issue importing it and I don't find any blog posts/more information about this module beside the public github repo

Machine Learning
ModelRiskApi
mrm
  • 1590 Views
  • 0 replies
  • 0 kudos
kashy
by New Contributor III
  • 11790 Views
  • 7 replies
  • 2 kudos

Resolved! Served model creation failed

I have a model registered in unity catalog which works fine and I can load / run and get results returned.  I wanted to create a serving endpoint but when I try I get this error.Served model creation failed for served model 'model_name', config versi...

  • 11790 Views
  • 7 replies
  • 2 kudos
Latest Reply
Annapurna_Hiriy
Databricks Employee
  • 2 kudos

 @kashy Looks like the model is not correctly referenced while loading. You should reference the path of the model till ‘model-best’, which is the top-level directory. loaded_model = mlflow.spacy.load_model("</path/to/your/model/>/model-best")    

  • 2 kudos
6 More Replies
Humi1245
by New Contributor
  • 3230 Views
  • 3 replies
  • 1 kudos

Cluster termination issue (It was working fine with same code previously)

Hi, I am running arima code on databricks commmunity edition. Last time, it worked but now the cluster gets terminated at this command and says its completed. but i cant further plot the predictions since the cluster is terminated.  # Fit the AutoARI...

  • 3230 Views
  • 3 replies
  • 1 kudos
Latest Reply
Tharun-Kumar
Databricks Employee
  • 1 kudos

@Humi1245  You can view the event log of the cluster to understand the cause of the cluster termination. If it is due to gc issues, then we have to understand the reason behind high driver memory utilization which caused the GC.  If it's due to inact...

  • 1 kudos
2 More Replies
Poised2Learn
by New Contributor III
  • 5356 Views
  • 2 replies
  • 1 kudos

Resolved! Model serving Ran out of memory

Hi fellows, I encountered memory(?) error when sending POST requests to my real-time endpoint, and I'm unable to find hardware setting to increase memory, as suggested by the Service Logs (below). Steps to Repro:(1) I registered a custom MLFlow model...

  • 5356 Views
  • 2 replies
  • 1 kudos
Latest Reply
Poised2Learn
New Contributor III
  • 1 kudos

Thank you for your responses, @Annapurna_Hiriy and @Retired_modIndeed, it appeared that my original model (~800MB) was too big for the current server. Based on your suggestion, I made a simpler/smaller model for this project, and then I was able to d...

  • 1 kudos
1 More Replies
Sujitha
by Databricks Employee
  • 4122 Views
  • 3 replies
  • 2 kudos

Resolved! Hello Community Users,  We recently announced a new Large Language Models (LLM) program, the first of its kind on edX! Learn how to develop production...

Hello Community Users, We recently announced a new Large Language Models (LLM) program, the first of its kind on edX! Learn how to develop production-ready LLM applications and dive into the theory behind foundation models. Taught by industry experts...

  • 4122 Views
  • 3 replies
  • 2 kudos
Latest Reply
APadmanabhan
Databricks Employee
  • 2 kudos

Hi @163050 You could download the Dbc file from the course, we already have the LLM course in the Customer Academy.

  • 2 kudos
2 More Replies
117074
by New Contributor III
  • 6321 Views
  • 1 replies
  • 1 kudos

sklearn logistic regression restarting kernel

Hello,I am trying to create a simple logistic regression model to test for the impact of PCA on my dataset. I'm getting the error in the attachment.It feels to me like a version mismatching - but I'm using a fresh cluster with no additional packages ...

  • 6321 Views
  • 1 replies
  • 1 kudos
Latest Reply
117074
New Contributor III
  • 1 kudos

Sorry for the long delay - holiday & work activities stopped from me progressing this issue. The error was actually an OOM error - fixed by configuring cluster sizes appropriately. 

  • 1 kudos
Soumyajeet_das
by New Contributor
  • 10849 Views
  • 2 replies
  • 0 kudos

Databricks ML Professional Certification exam got suspended due to technical issue

Hi Team,I was taking online exam for Databricks Machine Learning Professional exam on 22nd September,2023(16:15 Asia/Calcutta), the exam first got suspended due to proctoring issue and was rescheduled by the proctor in the next 30 mins, then the exam...

  • 10849 Views
  • 2 replies
  • 0 kudos
Latest Reply
Cert-Team
Databricks Employee
  • 0 kudos

@Soumyajeet_das Thank you for filing a ticket through support! The team is working on it. Thank you for your patience.

  • 0 kudos
1 More Replies
m12
by New Contributor II
  • 6543 Views
  • 2 replies
  • 2 kudos

Resolved! Enabling vector search in the workspace

Hi,I'm testing out LLM/RAG Databricks demo here: https://notebooks.databricks.com/demos/llm-rag-chatbot/index.html?_gl=1*1nj8hq2*_gcl_au*MTcxOTY0MDY4LjE2OTQ2MzgwNDU.# As part of the demo, I'm trying to create a vector search with the line below. vsc....

  • 6543 Views
  • 2 replies
  • 2 kudos
Latest Reply
Kumaran
Databricks Employee
  • 2 kudos

Hi @m12, Thank you for posting your question in the Databricks community. The vector search feature is currently undergoing a private preview. If you wish to participate, kindly complete the form provided below for onboarding. https://docs.google.com...

  • 2 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels