cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Science & Machine Learning

Forum Posts

enkefalos-commu
by New Contributor III
  • 995 Views
  • 4 replies
  • 0 kudos

Unable to create serving endpoint for the huggingface model phi-3-mini-128k-instruct

#20 69.92 ERROR: Could not find a version that satisfies the requirement transformers==4.41.0.dev0 (from versions: 0.1, 2.0.0, 2.1.0, 2.1.1, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.4.0, 2.4.1, 2.5.0, 2.5.1, 2.6.0, 2.7.0, 2.8.0, 2.9.0, 2.9.1, 2.10.0, 2.11.0, 3....

  • 995 Views
  • 4 replies
  • 0 kudos
Latest Reply
enkefalos-commu
New Contributor III
  • 0 kudos

@Kumaran I used latest one 2.12.1

  • 0 kudos
3 More Replies
kapwilson
by New Contributor II
  • 1107 Views
  • 1 replies
  • 1 kudos

Resolved! How to fine-tune OpenAI’s large language models (LLMs)

I am looking for the more detailed resources comparing RAG to fine-tuning methods in AI models to processing text data with LLM in laymen notes. I have found one resource but looking for the more  detailed view https://www.softwebsolutions.com/resour...

  • 1107 Views
  • 1 replies
  • 1 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @kapwilson, It seems you’re encountering an issue with using archive files in your Spark application submitted as a Jar task. Archive Files in Spark Applications: When submitting Spark applications, you can include additional files (such as Pyt...

  • 1 kudos
AndersenHuang
by New Contributor
  • 484 Views
  • 1 replies
  • 0 kudos

Spacy Retraining failure

Hello, I'm having problems trying to run my retraining notebook for a spacy model. The notebook creates a shell file with the following lines of code:    cmd = f''' awk '{{sub("source = ","source = /dbfs/FileStore/{dbfs_folder}/textcat/categories...

  • 484 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kumaran
Valued Contributor III
  • 0 kudos

Hi @AndersenHuang, Thank you for contacting Databricks community support. The error message you're encountering suggests that there's a permission issue when trying to copy the files. It's possible that the permissions for the directory /dbfs/FileSto...

  • 0 kudos
Edna
by New Contributor
  • 649 Views
  • 1 replies
  • 0 kudos

Model flavour using feature store model training log_model()

Hi I'm have succesfully registered my model using the feature engineering client with the following codes:with mlflow.start_run(): # Calculate the ratio of negative class samples to positive class samples ratio = (len(y_train) - y_train.sum()...

  • 649 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kumaran
Valued Contributor III
  • 0 kudos

Hello @Edna  Thank you for contacting Databricks community support. MLflow allows you to save models using different "flavors," which are essentially different ways of serializing and deserializing models. When you specify flavor=mlflow.sklearn, you'...

  • 0 kudos
tanjil
by New Contributor III
  • 890 Views
  • 3 replies
  • 0 kudos

Import mlflow Error

Hello, I am trying to replicate this motebook in my environment: mlflow-end-to-end-example - Databricks However, I am getting the following error when I run "import mlflow": "TypeError: bases must be types"How can I solve this issue? Thank you, Tanji...

  • 890 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kumaran
Valued Contributor III
  • 0 kudos

Hello @tanjil    Thank you for contacting databricks community support. Could you check what version of protobuf you have? If you are using 10.4 ML cluster, the MLflow 1.x is not compatible with protobuf 4.x. The default version of protobuf in MLR 10...

  • 0 kudos
2 More Replies
ml-engineer
by New Contributor
  • 384 Views
  • 1 replies
  • 0 kudos

while registering model I am getting error: AssertionError:

while registering model I am getting error: AssertionError:I am getting error while running the code with workflow if I running code individually with notebook then its running fine.  below is the code :            fe = FeatureEngineeringClient()    ...

  • 384 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kumaran
Valued Contributor III
  • 0 kudos

Hi @ml-engineer Thank you for contacting Databricks community support. I think you are installing tensorflow version 2.14, could you update to 2.16 using the pip install --upgrade tensorflow and see how it works?

  • 0 kudos
Intuit-AIHome
by New Contributor
  • 415 Views
  • 1 replies
  • 0 kudos

GenAI democratizes AI field

Interesting to hear the latest tech trend at Data +AI summit. 

  • 415 Views
  • 1 replies
  • 0 kudos
Latest Reply
artsheiko
Valued Contributor III
  • 0 kudos

you can find the synthesis on a DAIS 2023 announcements here : https://medium.com/@youssefmrini/data-and-ai-summit-2023-announcements-6a4aade1d54c

  • 0 kudos
Data_Cowboy
by New Contributor III
  • 1698 Views
  • 3 replies
  • 0 kudos

Resolved! Problems with xgboost.spark model loading from MLflow.

When loading an xgboost model from mlflow following the provided instructions in Databricks hosted MLflow the input sizes I am showing on the job are over 1 TB. Is anyone else using an xgboost.spark model and noticing the same behavior? Below are som...

image.png image
  • 1698 Views
  • 3 replies
  • 0 kudos
Latest Reply
dbx-user7354
New Contributor III
  • 0 kudos

Thank you very much @Data_Cowboy !!! I had the same issue. I even had 14 TiB  Databricks should really fix this

  • 0 kudos
2 More Replies
Colombia
by New Contributor II
  • 623 Views
  • 2 replies
  • 1 kudos

Use OF API from package enerbitdso 0.1.8 PYPI

Hello! I have code to use an API supplied in the energitdso package (This is the repository https://pypi.org/project/enerbitdso/). I changed the code adapting it to AZURE DATABRICKS in python, but although there is a connection with the API, it does ...

  • 623 Views
  • 2 replies
  • 1 kudos
Latest Reply
Colombia
New Contributor II
  • 1 kudos

The owner of the package updated it to use the time out as a parameter of up to 20 seconds and updated a dependent package in DataBricks, with the above the problem was solved

  • 1 kudos
1 More Replies
re
by New Contributor II
  • 515 Views
  • 2 replies
  • 0 kudos

RBAC and VectorSearch

When implementing the managed VectorSearch, what is the preferred way to implement row based access control? I see that you can use the filter API during a query, so simple filters using a certain column may work, but what if all the security informa...

  • 515 Views
  • 2 replies
  • 0 kudos
Latest Reply
re
New Contributor II
  • 0 kudos

Thanks AI for summarizing my question. However, you did not actually answer it.

  • 0 kudos
1 More Replies
moh3th1
by New Contributor
  • 529 Views
  • 1 replies
  • 0 kudos

Optimal Cluster Configuration for Training on Billion-Row Datasets

Hello Databricks Community,I am currently facing a challenge in configuring a cluster for training machine learning models on a dataset consisting of approximately a billion rows and 40 features. Given the volume of data, I want to ensure that the cl...

  • 529 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @moh3th1 ,  Machine Selection: Memory (RAM): Having sufficient memory is essential for large datasets. Ensure that your machine type has enough RAM to accommodate your data.CPU: CPU power impacts data processing speed. Consider CPUs with multiple...

  • 0 kudos
Anonymous
by Not applicable
  • 142467 Views
  • 60 replies
  • 5 kudos

Community Edition Login Issues Below is a list of troubleshooting steps for failing to login with email/password at community.cloud.databricks.com:   ...

Community Edition Login Issues   Below  is a list of troubleshooting steps for failing to login with email/password at community.cloud.databricks.com:       Troubleshooting Tips If this is your first time logging in, ensure that you did indeed sign u...

Image Image Image
  • 142467 Views
  • 60 replies
  • 5 kudos
Latest Reply
akuma67
New Contributor II
  • 5 kudos

Hey,I have been logged out and even the password reset email is not coming. How much time it takes to resolve?My account is ak.email86@gmail.com

  • 5 kudos
59 More Replies
amal15
by New Contributor II
  • 371 Views
  • 1 replies
  • 0 kudos

XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark ?

XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark ?How can I resolve this error?  

  • 371 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @amal15, The error message you’re encountering, “XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark,” indicates that the XGBoostEstimator class is not being recognized within the specified package.  Check Dependencie...

  • 0 kudos
e6exghu8
by New Contributor
  • 1031 Views
  • 1 replies
  • 0 kudos

Help - org.apache.spark.SparkException: Job aborted due to stage failure: Task 47 in stage 2842.0

Hello, I am training a SparkXGBRegressor model. It runs without errors if the complexity is low, however when I increase the max_depth and/or num_parallel_tree parameters, I get an error. I checked the cluster metrics during training and it doesn't l...

  • 1031 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @e6exghu8,  Ensure that your cluster has sufficient memory to handle the increased complexity (higher max_depth and num_parallel_tree).Check the memory configuration for your Spark executors. You might need to allocate more memory to each executor...

  • 0 kudos
cmilligan
by Contributor II
  • 4002 Views
  • 3 replies
  • 2 kudos

Issue with Multi-column In predicates are not supported in the DELETE condition.

I'm trying to delete rows from a table with the same date or id as records in another table. I'm using the below query and get the error 'Multi-column In predicates are not supported in the DELETE condition'. delete from cost_model.cm_dispatch_consol...

  • 4002 Views
  • 3 replies
  • 2 kudos
Latest Reply
shubhaskar
New Contributor II
  • 2 kudos

Had the same issue. Please check the subquery returned value there must be something wrong with that.

  • 2 kudos
2 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels