cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Science & Machine Learning

Forum Posts

ThomasSvane
by New Contributor
  • 869 Views
  • 1 replies
  • 0 kudos

Machine learning accuracy depends on execution plans

I'm using Databricks for a machine learning project -- a fairly standard text classification problem, where I want to use the description of an item (i.e. AXELTTNING KOLKERAMIK MM) to predict which of n product categories the item belongs to ('Bushin...

  • 869 Views
  • 1 replies
  • 0 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 0 kudos

that is weird.The regression algorithm should just do a prediction on a dataframe.  Such a huge difference in accuracy seems very suspicious.I would test the algorithm on a reference dataset, for which you know the accuracy beforehand.Perhaps your tr...

  • 0 kudos
Shreyash
by New Contributor II
  • 1645 Views
  • 5 replies
  • 0 kudos

java.lang.ClassNotFoundException: com.johnsnowlabs.nlp.DocumentAssembler

I am trying to serve a pyspark model using an endpoint. I was able to load and register the model normally. I could also load that model and perform inference but while serving the model, I am getting the following error: [94fffqts54] ERROR StatusLog...

Machine Learning
Model serving
sparknlp
  • 1645 Views
  • 5 replies
  • 0 kudos
Latest Reply
Rajora
New Contributor II
  • 0 kudos

I'm having the same problem and have tried various solutions with no luck. I found some potentially relevant information on the following link: https://www.johnsnowlabs.com/serving-spark-nlp-via-api-3-3-databricks-jobs-and-mlflow-serve-apis/  In the ...

  • 0 kudos
4 More Replies
tim-mcwilliams
by New Contributor III
  • 638 Views
  • 2 replies
  • 0 kudos

Notebook cell gets hung up but code completes

Have been running into an issue when running a pymc-marketing model in a Databricks notebook. The cell that fits the model gets hung up and the progress bar stops moving, however the code completes and dumps all needed output into a folder. After the...

  • 638 Views
  • 2 replies
  • 0 kudos
Latest Reply
tim-mcwilliams
New Contributor III
  • 0 kudos

Hi @Kaniz_Fatma , Thanks for the feedback here as well as on the other discussion forum. I've commented on your trouble shooting tips on that board. One thing to touch upon hereVerify that the output files are correct and contain the expected results...

  • 0 kudos
1 More Replies
elgeo
by Valued Contributor II
  • 2137 Views
  • 3 replies
  • 2 kudos

Table name as a parameter in SQL UDF

Hello experts,We would like to create a UDF function with input parameter a table_name. Please check the below simple example:CREATE OR REPLACE FUNCTION F_NAME(v_table_name STRING,                   v_w...

  • 2137 Views
  • 3 replies
  • 2 kudos
Latest Reply
alm
New Contributor III
  • 2 kudos

Did you find a solutions? I'm having the same problem

  • 2 kudos
2 More Replies
amal15
by New Contributor II
  • 979 Views
  • 2 replies
  • 0 kudos

error: not found: type XGBoostEstimator

error: not found: type XGBoostEstimator Spark & Scala  

  • 979 Views
  • 2 replies
  • 0 kudos
Latest Reply
shan_chandra
Esteemed Contributor
  • 0 kudos

@amal15 - can you please include the below to the import statement and see if it works. ml.dmlc.xgboost4j.scala.spark.XGBoostEstimator 

  • 0 kudos
1 More Replies
ccataV
by New Contributor
  • 1303 Views
  • 3 replies
  • 0 kudos

databricks-vectorsearch lib install

Hello,I'm trying to create and query a vector searc index like in this example : How to create and query a Vector Search index | Databricks on AWS on a databricks on azure. I have a cluster ina  private network so i need to install the suggested lib ...

  • 1303 Views
  • 3 replies
  • 0 kudos
Latest Reply
shan_chandra
Esteemed Contributor
  • 0 kudos

@ccataV - Databricks vector search is in public preview in selected regions as per the below documentation. However, since you look like a PVC user, you may need to reach out to Databricks support for accessibility. https://docs.databricks.com/en/gen...

  • 0 kudos
2 More Replies
Kaizen
by Contributor III
  • 1319 Views
  • 2 replies
  • 0 kudos

Model Serving Latency Chart

Hi, For the model serving latency graph what is p50 and p99? I only have one model i am serving on this endpoing so im surprised to see two models being tracked  

Kaizen_0-1714504038212.png
  • 1319 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaizen
Contributor III
  • 0 kudos

If im not mistaken this refers to 50% of responses and 99% responses and averages accordingly for the metrics? @s_park @Sujitha @Debayan 

  • 0 kudos
1 More Replies
VineethRavella
by New Contributor
  • 880 Views
  • 2 replies
  • 0 kudos

Databricks Conference

The dolly2.0 session involving RAGs was very informative!

  • 880 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @VineethRavella, Thank you for attending DAIS 2023 and for sharing your experience on the community platform. We're glad to hear that you found the Dolly2.0 session involving RAGs informative! We wanted to let you know that the Databricks Communit...

  • 0 kudos
1 More Replies
Prabhaker
by New Contributor III
  • 609 Views
  • 1 replies
  • 0 kudos

DataAI Summit 2023

Grean Learning and fun at the summit 2023 hosted by Databricks

  • 609 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Prabhaker,  It's great to hear that you had such a positive experience at DAIS 2023! We're thrilled that you found the summit to be a great learning and networking opportunity. We wanted to let you know that the Databricks community team will be ...

  • 0 kudos
HangLePhuonf
by New Contributor
  • 907 Views
  • 1 replies
  • 0 kudos

MLOPs

I’m here to learn more about DataBricks MLOps. I’ve learnt so much about how to build and maintain a production-level ML models. I will apply this knowledge to build a scalable ML solutions for my company. 

  • 907 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @HangLePhuonf ! Thank you for sharing your experience and enthusiasm for Databricks MLOps! It's great to hear that you found the sessions valuable and are planning to apply your learnings to build scalable ML solutions for your company. I wanted t...

  • 0 kudos
JamesBrown54
by New Contributor
  • 345 Views
  • 1 replies
  • 0 kudos

Retrieve data from multiple .mdb files using Python.

Hello,I'm interested in accessing several .mdb Access files stored in either Azure Data Lake Storage (ADLS) or the Databricks File System using Python. Could you provide guidance on how to accomplish this? It would be immensely helpful if you could a...

  • 345 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

  Hi @JamesBrown54, Let’s break down the steps for accessing .mdb Access files from both Azure data lake Storage (ADLS) and the Databricks File System using Python. Accessing .mdb Files in Azure data lake Storage (ADLS) Prerequisites: An Azure ...

  • 0 kudos
MinThuraZaw
by New Contributor III
  • 293 Views
  • 1 replies
  • 0 kudos

404 Page Not Found Error on Features page

We are facing this issue when accessing Features page. Our workspace is on AWS, ap-southeast-1.I think this is related to new feature for online tables and serverless. Is it because of online tables are not available yet in our region? If it not avai...

error2.png
  • 293 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @MinThuraZaw, I understand the issue you’re facing. Let’s break it down: Online Tables: Online Tables are a new feature in Databricks that allows you to create read-only copies of Delta tables optimized for online access. These tables are full...

  • 0 kudos
chrisf_sts
by New Contributor II
  • 574 Views
  • 1 replies
  • 0 kudos

Extract calculations naive bayes model

I have a naive Bayes ML model that takes call attributes and predicts if the caller is going to abandon the call while they are on hold waiting to speak to an agent. The model lives in Databricks ML flow, I have it registered. What I need to do is ex...

  • 574 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @chrisf_sts, Naive Bayes Model Overview: Naive Bayes is a probabilistic machine learning algorithm based on Bayes’ Theorem. It’s commonly used for classification tasks, such as spam filtering, document classification, and sentiment prediction....

  • 0 kudos
Leo69
by New Contributor
  • 519 Views
  • 1 replies
  • 0 kudos

"error_code":"INVALID_PARAMETER_VALUE","message":"INVALID_PARAMETER_VALUE: Failed to generate access

Hello everyone,I have an Azure Databricks subscription with my company, and I want to use external LLMs in databricks, like claude-3 or gemini. I managed to create a serving endpoint for Anthropic and I am able to use claude 3.But I want to use a Gem...

  • 519 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Leo69, It seems you’re encountering an issue while trying to use the Gemini model through Databricks. Let’s troubleshoot this together! First, let’s review some important information about external models in Databricks Model Serving. External...

  • 0 kudos
ryojikn
by New Contributor III
  • 334 Views
  • 1 replies
  • 0 kudos

Model Serving - Shadow Deployment - Azure

Hey,I'm composing an architecture within the usage of Model Serving Endpoints and one of the needs that we're aiming to resolve is Shadow Deployment.Currently, it seems that the traffic configurations available in model serving do not allow this type...

  • 334 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @ryojikn, Shadow deployment is a valuable strategy for testing and validating changes in a production environment without impacting live traffic. Let’s explore some approaches and resources related to shadow deployments: Shadow Deployments: Wh...

  • 0 kudos
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels