- 1754 Views
- 1 replies
- 0 kudos
GenAI democratizes AI field
Interesting to hear the latest tech trend at Data +AI summit.
- 1754 Views
- 1 replies
- 0 kudos
- 0 kudos
you can find the synthesis on a DAIS 2023 announcements here : https://medium.com/@youssefmrini/data-and-ai-summit-2023-announcements-6a4aade1d54c
- 0 kudos
- 7838 Views
- 3 replies
- 0 kudos
Resolved! Problems with xgboost.spark model loading from MLflow.
When loading an xgboost model from mlflow following the provided instructions in Databricks hosted MLflow the input sizes I am showing on the job are over 1 TB. Is anyone else using an xgboost.spark model and noticing the same behavior? Below are som...
- 7838 Views
- 3 replies
- 0 kudos
- 0 kudos
Thank you very much @Data_Cowboy !!! I had the same issue. I even had 14 TiB Databricks should really fix this
- 0 kudos
- 1762 Views
- 1 replies
- 0 kudos
Use OF API from package enerbitdso 0.1.8 PYPI
Hello! I have code to use an API supplied in the energitdso package (This is the repository https://pypi.org/project/enerbitdso/). I changed the code adapting it to AZURE DATABRICKS in python, but although there is a connection with the API, it does ...
- 1762 Views
- 1 replies
- 0 kudos
- 0 kudos
The owner of the package updated it to use the time out as a parameter of up to 20 seconds and updated a dependent package in DataBricks, with the above the problem was solved
- 0 kudos
- 3151 Views
- 1 replies
- 0 kudos
RBAC and VectorSearch
When implementing the managed VectorSearch, what is the preferred way to implement row based access control? I see that you can use the filter API during a query, so simple filters using a certain column may work, but what if all the security informa...
- 3151 Views
- 1 replies
- 0 kudos
- 0 kudos
Thanks AI for summarizing my question. However, you did not actually answer it.
- 0 kudos
- 2138 Views
- 0 replies
- 0 kudos
Optimal Cluster Configuration for Training on Billion-Row Datasets
Hello Databricks Community,I am currently facing a challenge in configuring a cluster for training machine learning models on a dataset consisting of approximately a billion rows and 40 features. Given the volume of data, I want to ensure that the cl...
- 2138 Views
- 0 replies
- 0 kudos
- 3941 Views
- 0 replies
- 0 kudos
Help - org.apache.spark.SparkException: Job aborted due to stage failure: Task 47 in stage 2842.0
Hello, I am training a SparkXGBRegressor model. It runs without errors if the complexity is low, however when I increase the max_depth and/or num_parallel_tree parameters, I get an error. I checked the cluster metrics during training and it doesn't l...
- 3941 Views
- 0 replies
- 0 kudos
- 7297 Views
- 2 replies
- 1 kudos
How to fix this runtime error in this Databricks distributed training tutorial workbook
I am following along with this notebook found from this article. I am attempting to fine tune the model with a single node and multiple GPUs, so I run everything up to the "Run Local Training" section, but from there I skip to "Run distributed traini...
- 7297 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi AChang, have you eventually resolved the error? I've also having the same error.
- 1 kudos
- 3619 Views
- 2 replies
- 1 kudos
Resolved! import ml.dmlc.xgboost4j.scala.spark.{XGBoostEstimator, XGBoostClassificationModel}
how i can import : import com.microsoft.ml.spark.{LightGBMClassifier,LightGBMClassificationModel}import ml.dmlc.xgboost4j.scala.spark.{XGBoostEstimator, XGBoostClassificationModel} projet spark & scala in databricks
- 3619 Views
- 2 replies
- 1 kudos
- 1 kudos
XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark ?How can I resolve this error?with maven : ml.dmlc:xgboost4j-spark_2.12:2.0.3
- 1 kudos
- 1171 Views
- 0 replies
- 0 kudos
XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark ?
XGBoostEstimator is not a member of package ml.dmlc.xgboost4j.scala.spark ?How can I resolve this error?
- 1171 Views
- 0 replies
- 0 kudos
- 1508 Views
- 0 replies
- 0 kudos
Extract calculations naive bayes model
I have a naive Bayes ML model that takes call attributes and predicts if the caller is going to abandon the call while they are on hold waiting to speak to an agent. The model lives in Databricks ML flow, I have it registered. What I need to do is ex...
- 1508 Views
- 0 replies
- 0 kudos
- 2873 Views
- 2 replies
- 0 kudos
Unity Catalog table management with multiple teams members
Hi! How are you guys managing large teams working on the same project. Each member has their own data to save in Unity Catalog.Based on my understanding there is only two ways to manage this:1) Create an individual member schea so they can store thei...
- 2873 Views
- 2 replies
- 0 kudos
- 1253 Views
- 0 replies
- 0 kudos
404 Page Not Found Error on Features page
We are facing this issue when accessing Features page. Our workspace is on AWS, ap-southeast-1.I think this is related to new feature for online tables and serverless. Is it because of online tables are not available yet in our region? If it not avai...
- 1253 Views
- 0 replies
- 0 kudos
- 6583 Views
- 5 replies
- 1 kudos
Resolved! Endpoint performance questions
Hi! Had really interesting results from some endpoint performance tests I did. I set up the non-optimized endpoint with zero-cluster scaling and optimized had this feature disabled.1) Why does the non-optimized endpoint have variable response time fo...
- 6583 Views
- 5 replies
- 1 kudos
- 1 kudos
Answering Q1: 1) The variable response time is due to the first endpoint response time requiring ~180 seconds to scale to 1 cluster from 02) Can i change zero scale time from the preset 30 min?
- 1 kudos
- 1922 Views
- 0 replies
- 0 kudos
Serving a custom transformer class via a pyfunc wrapper for a pyspark recommendation model
I am trying to serve an ALS pyspark model with a custom transformer(for generating user-specific recommendations) via a pyfunc wrapper. Although I can successfully score the logged model, the serving endpoint is throwing the following error.URI '/mod...
- 1922 Views
- 0 replies
- 0 kudos
- 49965 Views
- 25 replies
- 0 kudos
Problem when serving a langchain model on Databricks
I´m trying to model serving a LLM LangChain Model and every time it fails with this messsage:[6b6448zjll] [2024-02-06 14:09:55 +0000] [1146] [INFO] Booting worker with pid: 1146[6b6448zjll] An error occurred while loading the model. You haven't confi...
- 49965 Views
- 25 replies
- 0 kudos
- 0 kudos
Hi @DataWrangler and Team.I got to solve the initial problem from some tips you gave. I used your code as base and did some modifications adapted to what I have, I mean , No UC enabled and not able to use DatabricksEmbeddings, DatabricksVectorSearch ...
- 0 kudos
