cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Science & Machine Learning

Forum Posts

Marco98
by New Contributor II
  • 1372 Views
  • 1 replies
  • 2 kudos

Pytorch DDP on Databricks

Hello!I am currently trying to use Pytorch Lightning inside Databricks and I am currently using a cluster with 2 gpus. Whenever I try to train my Transformer model with 1 gpu in DP strategy everything works fine, but when I try to use both the 2 gpus...

  • 1372 Views
  • 1 replies
  • 2 kudos
Latest Reply
Chakra03
New Contributor II
  • 2 kudos

Hi @Marco Capusso​ , I am facing the similar issue could you find some fix. It would be great if you share some details around it.

  • 2 kudos
Direo
by Contributor
  • 1189 Views
  • 2 replies
  • 2 kudos

Feature store errors

HiWhen I open feature store, I get an error saying that "Failed to load some job schedules". When I open one of the feature store tables, I get several additional errors:"Failed to laod lates run for some job producers","Failed to laod some job produ...

image
  • 1189 Views
  • 2 replies
  • 2 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 2 kudos

Hi @Direo Direo​ , Just a friendly follow-up. Do you still need help, or @Hubert Dudek (Customer)​ 's response help you to find the solution? Please let us know.

  • 2 kudos
1 More Replies
venkat09
by New Contributor III
  • 1343 Views
  • 1 replies
  • 1 kudos

Resolved! Running into an issue while setting up dbx locally?

Followed the documentation and facing issue while running dbx execute on all-purpose/interactive cluster, which is up and running already. Ran this command  dbx execute --cluster-id=XXXXXX --job=dbx-demo-job --no-rebuild --debug. If anyone faced it ...

image
  • 1343 Views
  • 1 replies
  • 1 kudos
Latest Reply
venkat09
New Contributor III
  • 1 kudos

before running package it as wheel before running `dbx execute` fix the issue

  • 1 kudos
Nithin
by New Contributor II
  • 9200 Views
  • 14 replies
  • 4 kudos

Resolved! How to access databricks feature store outside databricks?

We are building the feature store using databricks API. Few of the machine learning engineers are using Jupyter notebooks. Is it possible to use feature store outside databricks?

  • 9200 Views
  • 14 replies
  • 4 kudos
Latest Reply
datariel
New Contributor II
  • 4 kudos

Hi @Kaniz Fatma​  and @Jose Gonzalez​ ,turning back to the original question, and considering that one of the main benefits of the Feature Store is the removal of the online/offline skew, how could I access to the features from a client application l...

  • 4 kudos
13 More Replies
naveen_marthala
by Contributor
  • 2882 Views
  • 4 replies
  • 2 kudos

Resolved! why does the client need to have git installed for auto-logging to an mlflow server running in "--serve-artifacts" mode?

I have an mlflow server with `--serve-artifacts` and with postgres as `--backend-store-uri`. The machine(container with base image python:3.9-bullseye) running the server has git installed which is available on path. I am logging from jupyter-noteboo...

  • 2882 Views
  • 4 replies
  • 2 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 2 kudos

Hi @Naveen Marthala​ ​ , Just a friendly follow-up. Do you still need help or the above responses help you to find the solution? Please let us know.

  • 2 kudos
3 More Replies
naveen_marthala
by Contributor
  • 5678 Views
  • 2 replies
  • 3 kudos

Resolved! How to PREVENT mlflow's autologging from logging ALL runs?

I am logging runs from jupyter notebook. the cells which has `mlflow.sklearn.autlog()` behaves as expected. but, the cells which has .fit() method being called on sklearn's estimators are also being logged as runs without explicitly mentioning `mlflo...

  • 5678 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

https://apkmiz.com/showbox-apk-old-latest-version-android/

  • 3 kudos
1 More Replies
Direo
by Contributor
  • 6180 Views
  • 4 replies
  • 2 kudos

Resolved! xgboost 1.5.1 gives 'XGBModel' object has no attribute 'enable_categorical' error

Should I pip install xgboost==1.4.2. (the last version it worked) or is there a better way to solve it having in mind that this solution might cause problems later if this version of xgboost is not supported on future python versions.

  • 6180 Views
  • 4 replies
  • 2 kudos
Latest Reply
Direo
Contributor
  • 2 kudos

Hi, @Kaniz Fatma​. No, I have found a solution. Needed to retrain models using new version of xgboost.

  • 2 kudos
3 More Replies
mradassaad
by New Contributor III
  • 2918 Views
  • 4 replies
  • 1 kudos

Resolved! Tuning `CrossValidator` spark job performance

I am running a 3-fold cross validation of an ML pipeline that utilizes `GBTClassifier` as the final step. It takes 18 hours to run and I am looking for feedback into how to improve the performance as I expect this to go faster.For context here is the...

Random Forest Job Random Forest Job Summary GBT storage top half
  • 2918 Views
  • 4 replies
  • 1 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @Assaad Mrad​  , Just a friendly follow-up. Do you still need help, or @Chris Chalcraft​ 's response help you to find the solution? Please let us know.

  • 1 kudos
3 More Replies
Mr__E
by Contributor II
  • 646 Views
  • 0 replies
  • 0 kudos

Custom AutoML evaluation metric for ranking model

I built a model which is used for ranking and I have a notebook that takes that model to generate rankings and then uses a UDF-based metric to evaluate those rankings. Is there any way that I can have this ranking / UDF be used during the AutoML trai...

  • 646 Views
  • 0 replies
  • 0 kudos
findinpath
by Contributor
  • 4009 Views
  • 9 replies
  • 5 kudos

How to mount s3 bucket in community edition cluster?

I'm using Databricks Community Edition for testing purposes on a OSS project.I'm spinning up the cluster automatically through Databricks Clusters API.The automated tests rely on AWS S3 infrastructure, reason why I need to mount the S3 bucket on the ...

  • 4009 Views
  • 9 replies
  • 5 kudos
Latest Reply
findinpath
Contributor
  • 5 kudos

I haven't found any solution.I'm assuming that currently my only option is the usage of Databricks Enterprise to model scenarios involving the mounting of object storage buckets.

  • 5 kudos
8 More Replies
Wayne
by New Contributor III
  • 1318 Views
  • 2 replies
  • 3 kudos
  • 1318 Views
  • 2 replies
  • 3 kudos
Latest Reply
Wayne
New Contributor III
  • 3 kudos

No error, just seeing the EXPAND DISK in cluster event logs. This is just a regular spark application. I am not sure if the cloud storage matters - a spark application uses it as input and output.

  • 3 kudos
1 More Replies
romanzdk
by New Contributor II
  • 1620 Views
  • 1 replies
  • 0 kudos

Databricks online store - Login to Azure SQL Database with Service Principal

I want to use Databricks Online Store with Azure SQL Database, however I am unable to autenthicate through Databricks Feature Store API. I need to use Service Principal credentials.I tried using Application ID as username and Secret as password, but ...

  • 1620 Views
  • 1 replies
  • 0 kudos
Latest Reply
romanzdk
New Contributor II
  • 0 kudos

no one?

  • 0 kudos
mhansinger
by New Contributor II
  • 13759 Views
  • 6 replies
  • 1 kudos

Resolved! Set default "spark.driver.maxResultSize" from the notebook

Hello,I would like to set the default "spark.driver.maxResultSize" from the notebook on my cluster. I know I can do that in the cluster settings, but is there a way to set it by code?I also know how to do it when I start a spark session, but in my ca...

  • 13759 Views
  • 6 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Maximilian Hansinger​ Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark the answer as best? If not, please tell us so we can help you.Thanks!

  • 1 kudos
5 More Replies
EdoardoVivo
by New Contributor
  • 1234 Views
  • 0 replies
  • 0 kudos

Pymc3 on Databricks: Progress bar

Hello everybody..I am trying to run pymc3 models on Databricks (runtime 9.1) and when I start the sampling process, the progress bar is not showing. It is a bit annoying since this way I do not have any information on when the process is going to end...

  • 1234 Views
  • 0 replies
  • 0 kudos
Vijeth
by New Contributor II
  • 3263 Views
  • 2 replies
  • 2 kudos

Resolved! How to deploy or create mlflow model as docker image with REST api endpoint within databricks?

Is it possible to create mlflow model as a docker image with REST api endpoint and use it for inferencing within databricks or hosting the image in azure container instances?

  • 3263 Views
  • 2 replies
  • 2 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 2 kudos

Hi @Vijeth Moudgalya​ ​ , Just a friendly follow-up. Did you follow @Bilal Aslam​ 's suggestion? Please let us know.

  • 2 kudos
1 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels