cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

tim-mcwilliams
by Visitor
  • 27 Views
  • 0 replies
  • 0 kudos

Notebook cell gets hung up but code completes

Have been running into an issue when running a pymc-marketing model in a Databricks notebook. The cell that fits the model gets hung up and the progress bar stops moving, however the code completes and dumps all needed output into a folder. After the...

  • 27 Views
  • 0 replies
  • 0 kudos
Data_Cowboy
by New Contributor III
  • 968 Views
  • 3 replies
  • 0 kudos

Resolved! Problems with xgboost.spark model loading from MLflow.

When loading an xgboost model from mlflow following the provided instructions in Databricks hosted MLflow the input sizes I am showing on the job are over 1 TB. Is anyone else using an xgboost.spark model and noticing the same behavior? Below are som...

image.png image
  • 968 Views
  • 3 replies
  • 0 kudos
Latest Reply
dbx-user7354
New Contributor III
  • 0 kudos

Thank you very much @Data_Cowboy !!! I had the same issue. I even had 14 TiB  Databricks should really fix this

  • 0 kudos
2 More Replies
Colombia
by New Contributor II
  • 273 Views
  • 2 replies
  • 1 kudos

Use OF API from package enerbitdso 0.1.8 PYPI

Hello! I have code to use an API supplied in the energitdso package (This is the repository https://pypi.org/project/enerbitdso/). I changed the code adapting it to AZURE DATABRICKS in python, but although there is a connection with the API, it does ...

  • 273 Views
  • 2 replies
  • 1 kudos
Latest Reply
Colombia
New Contributor II
  • 1 kudos

The owner of the package updated it to use the time out as a parameter of up to 20 seconds and updated a dependent package in DataBricks, with the above the problem was solved

  • 1 kudos
1 More Replies
re
by New Contributor II
  • 162 Views
  • 2 replies
  • 0 kudos

RBAC and VectorSearch

When implementing the managed VectorSearch, what is the preferred way to implement row based access control? I see that you can use the filter API during a query, so simple filters using a certain column may work, but what if all the security informa...

  • 162 Views
  • 2 replies
  • 0 kudos
Latest Reply
re
New Contributor II
  • 0 kudos

Thanks AI for summarizing my question. However, you did not actually answer it.

  • 0 kudos
1 More Replies
Lcsp
by New Contributor
  • 312 Views
  • 1 replies
  • 0 kudos

AssertionError Failed to create the catalog

getting this error when trying to setup the get-started-with-databricks-for-machine-learning  LAB . Unity catalog is enabled.  Validating the locally installed datasets: | listing local files...(0 seconds) | validation completed...(0 seconds total) C...

  • 312 Views
  • 1 replies
  • 0 kudos
Latest Reply
PL_db
New Contributor III
  • 0 kudos

 It looks like you don't have the CREATE CATALOG privilege on the metastore you're trying to create the catalog in: Privilege types by securable object in Unity Catalog 

  • 0 kudos
AndersenHuang
by New Contributor
  • 151 Views
  • 0 replies
  • 0 kudos

Spacy Retraining failure

Hello, I'm having problems trying to run my retraining notebook for a spacy model. The notebook creates a shell file with the following lines of code:    cmd = f''' awk '{{sub("source = ","source = /dbfs/FileStore/{dbfs_folder}/textcat/categories...

  • 151 Views
  • 0 replies
  • 0 kudos
moh3th1
by New Contributor
  • 111 Views
  • 1 replies
  • 0 kudos

Optimal Cluster Configuration for Training on Billion-Row Datasets

Hello Databricks Community,I am currently facing a challenge in configuring a cluster for training machine learning models on a dataset consisting of approximately a billion rows and 40 features. Given the volume of data, I want to ensure that the cl...

  • 111 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @moh3th1 ,  Machine Selection: Memory (RAM): Having sufficient memory is essential for large datasets. Ensure that your machine type has enough RAM to accommodate your data.CPU: CPU power impacts data processing speed. Consider CPUs with multiple...

  • 0 kudos
Anonymous
by Not applicable
  • 127948 Views
  • 60 replies
  • 3 kudos

Community Edition Login Issues Below is a list of troubleshooting steps for failing to login with email/password at community.cloud.databricks.com:   ...

Community Edition Login Issues   Below  is a list of troubleshooting steps for failing to login with email/password at community.cloud.databricks.com:       Troubleshooting Tips If this is your first time logging in, ensure that you did indeed sign u...

Image Image Image
  • 127948 Views
  • 60 replies
  • 3 kudos
Latest Reply
akuma67
New Contributor II
  • 3 kudos

Hey,I have been logged out and even the password reset email is not coming. How much time it takes to resolve?My account is ak.email86@gmail.com

  • 3 kudos
59 More Replies
Shreyash
by New Contributor II
  • 292 Views
  • 4 replies
  • 0 kudos

java.lang.ClassNotFoundException: com.johnsnowlabs.nlp.DocumentAssembler

I am trying to serve a pyspark model using an endpoint. I was able to load and register the model normally. I could also load that model and perform inference but while serving the model, I am getting the following error: [94fffqts54] ERROR StatusLog...

Machine Learning
Model serving
sparknlp
  • 292 Views
  • 4 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Shreyash, It looks like your code is encountering a java.lang.ClassNotFoundException for the com.johnsnowlabs.nlp.DocumentAssembler class while serving your PySpark model. This error occurs when the required class is not found in the classpath.  ...

  • 0 kudos
3 More Replies
amal15
by New Contributor II
  • 126 Views
  • 1 replies
  • 0 kudos

XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark ?

XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark ?How can I resolve this error?  

  • 126 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @amal15, The error message you’re encountering, “XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark,” indicates that the XGBoostEstimator class is not being recognized within the specified package.  Check Dependencie...

  • 0 kudos
e6exghu8
by New Contributor
  • 312 Views
  • 1 replies
  • 0 kudos

Help - org.apache.spark.SparkException: Job aborted due to stage failure: Task 47 in stage 2842.0

Hello, I am training a SparkXGBRegressor model. It runs without errors if the complexity is low, however when I increase the max_depth and/or num_parallel_tree parameters, I get an error. I checked the cluster metrics during training and it doesn't l...

  • 312 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @e6exghu8,  Ensure that your cluster has sufficient memory to handle the increased complexity (higher max_depth and num_parallel_tree).Check the memory configuration for your Spark executors. You might need to allocate more memory to each executor...

  • 0 kudos
cmilligan
by Contributor II
  • 3154 Views
  • 3 replies
  • 2 kudos

Issue with Multi-column In predicates are not supported in the DELETE condition.

I'm trying to delete rows from a table with the same date or id as records in another table. I'm using the below query and get the error 'Multi-column In predicates are not supported in the DELETE condition'. delete from cost_model.cm_dispatch_consol...

  • 3154 Views
  • 3 replies
  • 2 kudos
Latest Reply
shubhaskar
New Contributor II
  • 2 kudos

Had the same issue. Please check the subquery returned value there must be something wrong with that.

  • 2 kudos
2 More Replies
AChang
by New Contributor III
  • 1906 Views
  • 2 replies
  • 1 kudos

How to fix this runtime error in this Databricks distributed training tutorial workbook

I am following along with this notebook found from this article. I am attempting to fine tune the model with a single node and multiple GPUs, so I run everything up to the "Run Local Training" section, but from there I skip to "Run distributed traini...

  • 1906 Views
  • 2 replies
  • 1 kudos
Latest Reply
KYX
New Contributor II
  • 1 kudos

Hi AChang, have you eventually resolved the error? I've also having the same error.

  • 1 kudos
1 More Replies
Labels