cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

enkefalos-commu
by New Contributor III
  • 464 Views
  • 4 replies
  • 0 kudos

Unable to create serving endpoint for the huggingface model phi-3-mini-128k-instruct

#20 69.92 ERROR: Could not find a version that satisfies the requirement transformers==4.41.0.dev0 (from versions: 0.1, 2.0.0, 2.1.0, 2.1.1, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.4.0, 2.4.1, 2.5.0, 2.5.1, 2.6.0, 2.7.0, 2.8.0, 2.9.0, 2.9.1, 2.10.0, 2.11.0, 3....

  • 464 Views
  • 4 replies
  • 0 kudos
Latest Reply
enkefalos-commu
New Contributor III
  • 0 kudos

@Kumaran I used latest one 2.12.1

  • 0 kudos
3 More Replies
kapwilson
by New Contributor II
  • 462 Views
  • 1 replies
  • 1 kudos

Resolved! How to fine-tune OpenAI’s large language models (LLMs)

I am looking for the more detailed resources comparing RAG to fine-tuning methods in AI models to processing text data with LLM in laymen notes. I have found one resource but looking for the more  detailed view https://www.softwebsolutions.com/resour...

  • 462 Views
  • 1 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @kapwilson, It seems you’re encountering an issue with using archive files in your Spark application submitted as a Jar task. Archive Files in Spark Applications: When submitting Spark applications, you can include additional files (such as Pyt...

  • 1 kudos
AndersenHuang
by New Contributor
  • 270 Views
  • 1 replies
  • 0 kudos

Spacy Retraining failure

Hello, I'm having problems trying to run my retraining notebook for a spacy model. The notebook creates a shell file with the following lines of code:    cmd = f''' awk '{{sub("source = ","source = /dbfs/FileStore/{dbfs_folder}/textcat/categories...

  • 270 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kumaran
Valued Contributor III
  • 0 kudos

Hi @AndersenHuang, Thank you for contacting Databricks community support. The error message you're encountering suggests that there's a permission issue when trying to copy the files. It's possible that the permissions for the directory /dbfs/FileSto...

  • 0 kudos
Edna
by New Contributor
  • 430 Views
  • 1 replies
  • 0 kudos

Model flavour using feature store model training log_model()

Hi I'm have succesfully registered my model using the feature engineering client with the following codes:with mlflow.start_run(): # Calculate the ratio of negative class samples to positive class samples ratio = (len(y_train) - y_train.sum()...

  • 430 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kumaran
Valued Contributor III
  • 0 kudos

Hello @Edna  Thank you for contacting Databricks community support. MLflow allows you to save models using different "flavors," which are essentially different ways of serializing and deserializing models. When you specify flavor=mlflow.sklearn, you'...

  • 0 kudos
tanjil
by New Contributor III
  • 531 Views
  • 3 replies
  • 0 kudos

Import mlflow Error

Hello, I am trying to replicate this motebook in my environment: mlflow-end-to-end-example - Databricks However, I am getting the following error when I run "import mlflow": "TypeError: bases must be types"How can I solve this issue? Thank you, Tanji...

  • 531 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kumaran
Valued Contributor III
  • 0 kudos

Hello @tanjil    Thank you for contacting databricks community support. Could you check what version of protobuf you have? If you are using 10.4 ML cluster, the MLflow 1.x is not compatible with protobuf 4.x. The default version of protobuf in MLR 10...

  • 0 kudos
2 More Replies
ml-engineer
by New Contributor
  • 196 Views
  • 1 replies
  • 0 kudos

while registering model I am getting error: AssertionError:

while registering model I am getting error: AssertionError:I am getting error while running the code with workflow if I running code individually with notebook then its running fine.  below is the code :            fe = FeatureEngineeringClient()    ...

  • 196 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kumaran
Valued Contributor III
  • 0 kudos

Hi @ml-engineer Thank you for contacting Databricks community support. I think you are installing tensorflow version 2.14, could you update to 2.16 using the pip install --upgrade tensorflow and see how it works?

  • 0 kudos
Intuit-AIHome
by New Contributor
  • 302 Views
  • 1 replies
  • 0 kudos

GenAI democratizes AI field

Interesting to hear the latest tech trend at Data +AI summit. 

  • 302 Views
  • 1 replies
  • 0 kudos
Latest Reply
artsheiko
Valued Contributor III
  • 0 kudos

you can find the synthesis on a DAIS 2023 announcements here : https://medium.com/@youssefmrini/data-and-ai-summit-2023-announcements-6a4aade1d54c

  • 0 kudos
Data_Cowboy
by New Contributor III
  • 1265 Views
  • 3 replies
  • 0 kudos

Resolved! Problems with xgboost.spark model loading from MLflow.

When loading an xgboost model from mlflow following the provided instructions in Databricks hosted MLflow the input sizes I am showing on the job are over 1 TB. Is anyone else using an xgboost.spark model and noticing the same behavior? Below are som...

image.png image
  • 1265 Views
  • 3 replies
  • 0 kudos
Latest Reply
dbx-user7354
New Contributor III
  • 0 kudos

Thank you very much @Data_Cowboy !!! I had the same issue. I even had 14 TiB  Databricks should really fix this

  • 0 kudos
2 More Replies
Colombia
by New Contributor II
  • 440 Views
  • 2 replies
  • 1 kudos

Use OF API from package enerbitdso 0.1.8 PYPI

Hello! I have code to use an API supplied in the energitdso package (This is the repository https://pypi.org/project/enerbitdso/). I changed the code adapting it to AZURE DATABRICKS in python, but although there is a connection with the API, it does ...

  • 440 Views
  • 2 replies
  • 1 kudos
Latest Reply
Colombia
New Contributor II
  • 1 kudos

The owner of the package updated it to use the time out as a parameter of up to 20 seconds and updated a dependent package in DataBricks, with the above the problem was solved

  • 1 kudos
1 More Replies
re
by New Contributor II
  • 311 Views
  • 2 replies
  • 0 kudos

RBAC and VectorSearch

When implementing the managed VectorSearch, what is the preferred way to implement row based access control? I see that you can use the filter API during a query, so simple filters using a certain column may work, but what if all the security informa...

  • 311 Views
  • 2 replies
  • 0 kudos
Latest Reply
re
New Contributor II
  • 0 kudos

Thanks AI for summarizing my question. However, you did not actually answer it.

  • 0 kudos
1 More Replies
moh3th1
by New Contributor
  • 248 Views
  • 1 replies
  • 0 kudos

Optimal Cluster Configuration for Training on Billion-Row Datasets

Hello Databricks Community,I am currently facing a challenge in configuring a cluster for training machine learning models on a dataset consisting of approximately a billion rows and 40 features. Given the volume of data, I want to ensure that the cl...

  • 248 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @moh3th1 ,  Machine Selection: Memory (RAM): Having sufficient memory is essential for large datasets. Ensure that your machine type has enough RAM to accommodate your data.CPU: CPU power impacts data processing speed. Consider CPUs with multiple...

  • 0 kudos
Anonymous
by Not applicable
  • 133508 Views
  • 60 replies
  • 3 kudos

Community Edition Login Issues Below is a list of troubleshooting steps for failing to login with email/password at community.cloud.databricks.com:   ...

Community Edition Login Issues   Below  is a list of troubleshooting steps for failing to login with email/password at community.cloud.databricks.com:       Troubleshooting Tips If this is your first time logging in, ensure that you did indeed sign u...

Image Image Image
  • 133508 Views
  • 60 replies
  • 3 kudos
Latest Reply
akuma67
New Contributor II
  • 3 kudos

Hey,I have been logged out and even the password reset email is not coming. How much time it takes to resolve?My account is ak.email86@gmail.com

  • 3 kudos
59 More Replies
amal15
by New Contributor II
  • 244 Views
  • 1 replies
  • 0 kudos

XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark ?

XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark ?How can I resolve this error?  

  • 244 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @amal15, The error message you’re encountering, “XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark,” indicates that the XGBoostEstimator class is not being recognized within the specified package.  Check Dependencie...

  • 0 kudos
e6exghu8
by New Contributor
  • 560 Views
  • 1 replies
  • 0 kudos

Help - org.apache.spark.SparkException: Job aborted due to stage failure: Task 47 in stage 2842.0

Hello, I am training a SparkXGBRegressor model. It runs without errors if the complexity is low, however when I increase the max_depth and/or num_parallel_tree parameters, I get an error. I checked the cluster metrics during training and it doesn't l...

  • 560 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @e6exghu8,  Ensure that your cluster has sufficient memory to handle the increased complexity (higher max_depth and num_parallel_tree).Check the memory configuration for your Spark executors. You might need to allocate more memory to each executor...

  • 0 kudos
cmilligan
by Contributor II
  • 3537 Views
  • 3 replies
  • 2 kudos

Issue with Multi-column In predicates are not supported in the DELETE condition.

I'm trying to delete rows from a table with the same date or id as records in another table. I'm using the below query and get the error 'Multi-column In predicates are not supported in the DELETE condition'. delete from cost_model.cm_dispatch_consol...

  • 3537 Views
  • 3 replies
  • 2 kudos
Latest Reply
shubhaskar
New Contributor II
  • 2 kudos

Had the same issue. Please check the subquery returned value there must be something wrong with that.

  • 2 kudos
2 More Replies
Labels