cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Science & Machine Learning

Forum Posts

Sujitha
by Community Manager
  • 5413 Views
  • 0 replies
  • 0 kudos

Ready to elevate your data insights to new heights? Discover the power of Databricks SQL Materialize

In our latest blog post, unlock the secrets to delivering fresh data and actionable insights straight to your business. Learn how Lakeview dashboards, now available on the Databricks Data Intelligence Platform, revolutionize visualization and reporti...

Screenshot 2024-02-06 at 7.06.45 PM.png
  • 5413 Views
  • 0 replies
  • 0 kudos
sergio-calderon
by New Contributor II
  • 799 Views
  • 3 replies
  • 0 kudos

How to use Databricks secrets on MLFlow conda dependencies?

Hi!Do you know if it's correct to use the plain user and token for installing a custom dependency (an internal python package) in a mlflow registered model? (it's the only way I get it working because if not it can't install the dependency) It works,...

  • 799 Views
  • 3 replies
  • 0 kudos
Latest Reply
sergio-calderon
New Contributor II
  • 0 kudos

Thank you very much for the response but using that way is the same as the plain text approach, right? I mean, you are writing it from a Notebook and that's what I've done, but if you open the .yaml file the values are there with plain text.

  • 0 kudos
2 More Replies
NDK_1
by New Contributor II
  • 3250 Views
  • 6 replies
  • 2 kudos

Salesforce connection with Databricks

How can we connect with salesforce from databricks without using any third party jar files?

  • 3250 Views
  • 6 replies
  • 2 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 2 kudos

That’s a great tip! Thank you for sharing your knowledge with the community. I’m sure many people will find it helpful. Keep up the good work!

  • 2 kudos
5 More Replies
VeereshKH
by New Contributor II
  • 1742 Views
  • 4 replies
  • 0 kudos

org.apache.spark.SparkException: Job aborted due to stage failure during Model Training

org.apache.spark.SparkException: Job aborted due to stage failure: Could not recover from a failed barrier ResultStage. Most recent failure reason: Stage failed because barrier task ResultTask(160, 13) finished unsuccessfully.

  • 1742 Views
  • 4 replies
  • 0 kudos
Latest Reply
Lakshay
Esteemed Contributor
  • 0 kudos

Could you share the stage details where the issue happened?

  • 0 kudos
3 More Replies
VeereshKH
by New Contributor II
  • 828 Views
  • 2 replies
  • 0 kudos

Synapse ML - TypeError: 'JavaPackage' object is not callable

from synapse.ml.lightgbm import *LightGBMRegressor()gives me TypeError: 'JavaPackage' object is not callableDBR 8.2 (includes Apache Spark 3.1.1, Scala 2.12)

  • 828 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hey there! Thanks a bunch for being part of our awesome community!  We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solution...

  • 0 kudos
1 More Replies
yhyhy3
by New Contributor III
  • 682 Views
  • 1 replies
  • 0 kudos

Foundation Model APIs HIPAA compliance

I saw that Foundation Model API  is not HIPAA compliant. Is there a timeline in which we could expect it to be HIPAA compliant? I work for a healthcare company with a BAA with Databricks.

  • 682 Views
  • 1 replies
  • 0 kudos
Latest Reply
saikumar246
New Contributor III
  • 0 kudos

Hi @yhyhy3  Foundation Model API's HIPAA certification:AWS: e.t.a. March 2024Azure: e.t.a. Aug 2024 HIPAA certification is essentially having a third party audit report for HIPAA.  That is not the date that a HIPAA product offering may/will necessari...

  • 0 kudos
pablobd
by Contributor
  • 941 Views
  • 2 replies
  • 0 kudos

Update model serving endpoint

Hi all,I've been able to create a model serving endpoint through the api, following the docs, but then when trying to update the version, I get the following error:'{"error_code":"RESOURCE_ALREADY_EXISTS","message":"Endpoint with name \'ml-project\' ...

  • 941 Views
  • 2 replies
  • 0 kudos
Latest Reply
BR_DatabricksAI
Contributor
  • 0 kudos

Folks, Alternate way you can also deploy the models in serving layer with different versions. Though I am using mLflow. You may also refer to the below link if it its helpful How to Quickly Deploy, Test & Manage ML Models as REST Endpoints with Datab...

  • 0 kudos
1 More Replies
pablobd
by Contributor
  • 2843 Views
  • 5 replies
  • 2 kudos

Resolved! Error when accessing rdd of DataFrame

I need to run this kind of code: from pyspark.sql import SparkSession import pandas as pd # Create a Spark session spark = SparkSession.builder.appName("example").getOrCreate() # Sample data data = [("Alice", 1), ("Bob", 2), ("Charlie", 3), ("David...

  • 2843 Views
  • 5 replies
  • 2 kudos
Latest Reply
pablobd
Contributor
  • 2 kudos

I found a good solution that works both locally and in the cloud. Copy pasting the code in case it helps someone.This is the higher level function in charge of partitioning the data and sending the data and the function fn to each node. def decrypt_d...

  • 2 kudos
4 More Replies
BalaRamesh
by New Contributor II
  • 2407 Views
  • 7 replies
  • 1 kudos

Getting Permission Denied on model

while creating service endpoint , getting permission denied error. Need help how to provide the permission.  

BalaRamesh_0-1704953863222.png
  • 2407 Views
  • 7 replies
  • 1 kudos
Latest Reply
Adheer
New Contributor II
  • 1 kudos

@Debayan I managed to solve the issue using databricks sdk library. From UI it was failing with the same error as mentioned by @BalaRamesh 

  • 1 kudos
6 More Replies
Kaizen
by Contributor III
  • 1945 Views
  • 5 replies
  • 0 kudos

Cant access path for Shared Access Mode Cluster

Hi! I was able to successfully download and run selenium on both single and no isolation mode clusters. But the shared cluster do not seem to have permission to access the path of drivers needed such as chrome and chromedriver.1) what is the general ...

Kaizen_0-1705688464489.png Kaizen_1-1705688529116.png
  • 1945 Views
  • 5 replies
  • 0 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 0 kudos

Hi, Is the cluster UC enabled? Also, what are the types of the drivers (chrome and chromedriver) ? 

  • 0 kudos
4 More Replies
Noura_azza
by New Contributor II
  • 559 Views
  • 1 replies
  • 0 kudos

AutoML split with dt column not working properly

I am using AutoML and want to split my data to train/validation and test  using a dt column (one date for train one different date for validation and a third date for test. The problem that the autoML fails, there are only training metrics (no valiat...

  • 559 Views
  • 1 replies
  • 0 kudos
Latest Reply
Noura_azza
New Contributor II
  • 0 kudos

this is what I see in my data exploration  notebook. All dates are considered part of the training split   

  • 0 kudos
tessaickx
by New Contributor III
  • 1920 Views
  • 2 replies
  • 0 kudos

Serving endpoints: model server failed to load the model: the file bash was not found: uknown

While trying to create a serving endpoint with my custom model, I get a "Failed" state:Model server failed to load the model. Please see service logs for more information.The service logs show the following:Container failed with: failed to create con...

  • 1920 Views
  • 2 replies
  • 0 kudos
Latest Reply
ravi-malipeddi
New Contributor II
  • 0 kudos

I have faced the similar issue. still didn't find the right solution. In my case, the below is the error trace i found from service logs. Not sure where the issue could be"An error occurred while loading the model. You haven't configured the CLI yet!...

  • 0 kudos
1 More Replies
pablobd
by Contributor
  • 1398 Views
  • 2 replies
  • 1 kudos

Resolved! Model Serving Endpoint Creation through API

Hello,I am trying to create a model serving endpoint via the API as explained here: https://docs.databricks.com/api/workspace/servingendpoints/createI created a trusted IAM role with access to DynamoDB for the feature store. I try to use this field,"...

  • 1398 Views
  • 2 replies
  • 1 kudos
Latest Reply
pablobd
Contributor
  • 1 kudos

Thanks @Debayan - this article helped

  • 1 kudos
1 More Replies
leelee3000
by New Contributor III
  • 1155 Views
  • 2 replies
  • 1 kudos

Databricks OpenAI Integration Issue

Facing challenges in leveraging Databricks for serving, logging, and monitoring OpenAI usage within an Azure environment. Specifically, encountering issues with Inference Tables not being enabled in the UI when creating serving endpoints with externa...

  • 1155 Views
  • 2 replies
  • 1 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 1 kudos

Hi, In addition to the previous message, you can refer to https://learn.microsoft.com/en-us/azure/databricks/large-language-models/ai-generate-text-example. 

  • 1 kudos
1 More Replies
BR_DatabricksAI
by Contributor
  • 1130 Views
  • 3 replies
  • 1 kudos

Unable to distribute the workload to different worker

Hello Team, I am unable to distribute the workload to databricks different worker while using the hugging face GPT2 LLM model. Jobs always use the 1 node even though we have the min and max worker node setting with 2. Appreciate if anyone can share a...

  • 1130 Views
  • 3 replies
  • 1 kudos
Latest Reply
BR_DatabricksAI
Contributor
  • 1 kudos

Hello, We have around between 5k to 10K transcript files available in the ADLS gen 2 and we are using hugging face gpt2 model to train and server the model and expecting to pass the workload to different cluster nodes while serving the LLM model and ...

  • 1 kudos
2 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels