- 4790 Views
- 1 replies
- 0 kudos
RuntimeError: Expected to mark a variable ready only once error
I'm using a Single Node machine with g5-2x-large to fine tune a LLaMa-2 model. My Come Notebook runs very smoothly on Google Col but when I try to run it on `Databricks`, it throws me the exact error given below:RuntimeError: Expected to mark a varia...
- 4790 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @saleem_shady! Have you tried including the parameter ddp_find_unused_parameters=False in your TrainingArguments? Here's an example of how to include it: https://github.com/databricks/databricks-ml-examples/blob/master/llm-models/llamav2/llamav...
- 0 kudos
- 1702 Views
- 1 replies
- 0 kudos
Specify extras of library in asset bundles yml file
Hello,I am using asset bundles to build, deploy, create a cluster and install a package in it. The package comes with some extra dependencies that I also want to include in the cluster. Do you know how to specify this extras in the databricks.yml fil...
- 1702 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @pablobd! Have you tried declaring these additional dependencies of your wheel file before the wheel file itself? Here's an example snippet for clarification: resources: jobs: train: name: train-ml tasks: - task_key: train-ml ...
- 0 kudos
- 2206 Views
- 3 replies
- 0 kudos
How to I select an 80/10/10 split when doing AutoML
Headline says it all. I am doing a regression and want to select a testvaltrain split that is not 60/20/20. Anyone know how to do this?
- 2206 Views
- 3 replies
- 0 kudos
- 0 kudos
You'd need to put 80% of your data with the earliest timestamp, then 10% with another one and 10% with another.
- 0 kudos
- 1736 Views
- 0 replies
- 1 kudos
Machine Learning Practitioner learning Plan Notebook demos
I am enrolled on the Machine Learning Practitioner learning Plan free version, I can't get the notebook demos to run on databricks community edition. How can I do the demo practices of these courses? Is there another alternative?
- 1736 Views
- 0 replies
- 1 kudos
- 2508 Views
- 1 replies
- 0 kudos
Machine learning accuracy depends on execution plans
I'm using Databricks for a machine learning project -- a fairly standard text classification problem, where I want to use the description of an item (i.e. AXELTTNING KOLKERAMIK MM) to predict which of n product categories the item belongs to ('Bushin...
- 2508 Views
- 1 replies
- 0 kudos
- 0 kudos
that is weird.The regression algorithm should just do a prediction on a dataframe. Such a huge difference in accuracy seems very suspicious.I would test the algorithm on a reference dataset, for which you know the accuracy beforehand.Perhaps your tr...
- 0 kudos
- 6358 Views
- 3 replies
- 0 kudos
java.lang.ClassNotFoundException: com.johnsnowlabs.nlp.DocumentAssembler
I am trying to serve a pyspark model using an endpoint. I was able to load and register the model normally. I could also load that model and perform inference but while serving the model, I am getting the following error: [94fffqts54] ERROR StatusLog...
- 6358 Views
- 3 replies
- 0 kudos
- 0 kudos
I'm having the same problem and have tried various solutions with no luck. I found some potentially relevant information on the following link: https://www.johnsnowlabs.com/serving-spark-nlp-via-api-3-3-databricks-jobs-and-mlflow-serve-apis/ In the ...
- 0 kudos
- 2328 Views
- 1 replies
- 0 kudos
Notebook cell gets hung up but code completes
Have been running into an issue when running a pymc-marketing model in a Databricks notebook. The cell that fits the model gets hung up and the progress bar stops moving, however the code completes and dumps all needed output into a folder. After the...
- 2328 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Retired_mod , Thanks for the feedback here as well as on the other discussion forum. I've commented on your trouble shooting tips on that board. One thing to touch upon hereVerify that the output files are correct and contain the expected results...
- 0 kudos
- 1870 Views
- 0 replies
- 0 kudos
Content Type error legacy serving
Hi,I have deployed an endpoint in Databricks using legacy serving. I am using the custom pyfunc in mlflow to deploy the custom code. This code uses Machine Learning to parse out the table of contents in some pdf files then returns the table of conten...
- 1870 Views
- 0 replies
- 0 kudos
- 5605 Views
- 3 replies
- 2 kudos
Table name as a parameter in SQL UDF
Hello experts,We would like to create a UDF function with input parameter a table_name. Please check the below simple example:CREATE OR REPLACE FUNCTION F_NAME(v_table_name STRING, v_w...
- 5605 Views
- 3 replies
- 2 kudos
- 2289 Views
- 1 replies
- 0 kudos
error: not found: type XGBoostEstimator
error: not found: type XGBoostEstimator Spark & Scala
- 2289 Views
- 1 replies
- 0 kudos
- 0 kudos
@amal15 - can you please include the below to the import statement and see if it works. ml.dmlc.xgboost4j.scala.spark.XGBoostEstimator
- 0 kudos
- 5119 Views
- 2 replies
- 0 kudos
databricks-vectorsearch lib install
Hello,I'm trying to create and query a vector searc index like in this example : How to create and query a Vector Search index | Databricks on AWS on a databricks on azure. I have a cluster ina private network so i need to install the suggested lib ...
- 5119 Views
- 2 replies
- 0 kudos
- 0 kudos
@ccataV - Databricks vector search is in public preview in selected regions as per the below documentation. However, since you look like a PVC user, you may need to reach out to Databricks support for accessibility. https://docs.databricks.com/en/gen...
- 0 kudos
- 4469 Views
- 2 replies
- 0 kudos
Model Serving Latency Chart
Hi, For the model serving latency graph what is p50 and p99? I only have one model i am serving on this endpoing so im surprised to see two models being tracked
- 4469 Views
- 2 replies
- 0 kudos
- 0 kudos
If im not mistaken this refers to 50% of responses and 99% responses and averages accordingly for the metrics? @s_park @Sujitha @Debayan
- 0 kudos
- 5556 Views
- 3 replies
- 0 kudos
Unable to create serving endpoint for the huggingface model phi-3-mini-128k-instruct
#20 69.92 ERROR: Could not find a version that satisfies the requirement transformers==4.41.0.dev0 (from versions: 0.1, 2.0.0, 2.1.0, 2.1.1, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 2.4.0, 2.4.1, 2.5.0, 2.5.1, 2.6.0, 2.7.0, 2.8.0, 2.9.0, 2.9.1, 2.10.0, 2.11.0, 3....
- 5556 Views
- 3 replies
- 0 kudos
- 2553 Views
- 0 replies
- 0 kudos
How to fine-tune OpenAI’s large language models (LLMs)
I am looking for the more detailed resources comparing RAG to fine-tuning methods in AI models to processing text data with LLM in laymen notes. I have found one resource but looking for the more detailed view https://www.softwebsolutions.com/resour...
- 2553 Views
- 0 replies
- 0 kudos
- 1664 Views
- 1 replies
- 0 kudos
Spacy Retraining failure
Hello, I'm having problems trying to run my retraining notebook for a spacy model. The notebook creates a shell file with the following lines of code: cmd = f''' awk '{{sub("source = ","source = /dbfs/FileStore/{dbfs_folder}/textcat/categories...
- 1664 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @AndersenHuang, Thank you for contacting Databricks community support. The error message you're encountering suggests that there's a permission issue when trying to copy the files. It's possible that the permissions for the directory /dbfs/FileSto...
- 0 kudos
