cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

108387
by New Contributor II
  • 13252 Views
  • 1 replies
  • 1 kudos

Intermittent "Unexpected failure while waiting for the cluster to be ready" Error

We are encountering an occasional issue where jobs may fail with a message like the following:Run result unavailable: job failed with error message Unexpected failure while waiting for the cluster (ID) to be ready.Cause Unexpected state for cluster (...

  • 13252 Views
  • 1 replies
  • 1 kudos
MattM
by New Contributor III
  • 1838 Views
  • 1 replies
  • 2 kudos

Mapping Control data - Maintained by Business User

We are ingesting our data from ADLS into databricks as delta table. After raw layer we need to refer to a control\mapping layer which defines certain logic\measure definition. This would be incorporated in the subsequent silver or gold layer. This co...

  • 1838 Views
  • 1 replies
  • 2 kudos
Latest Reply
MattM
New Contributor III
  • 2 kudos

Thanks for your response. Can business user without the help of any script modify any rows in the table after loading it onetime from CSV fiels?

  • 2 kudos
vaibhavsingal11
by New Contributor
  • 4051 Views
  • 3 replies
  • 4 kudos

Error executing mlflow's mlflow.register_model() in applyInPandas to register Model to another workspace.

Hi Team,I am getting the following error (mlflow.utils.databricks_utils._NoDbutilsError) while trying to register model using applyInPandas to another databricks workspace. I have already set the secret scope and prefix to authenticate against the 2n...

  • 4051 Views
  • 3 replies
  • 4 kudos
Latest Reply
Amardeep
New Contributor II
  • 4 kudos

@vaibhavsingal11​ , Hey, any progress on this? I'm facing the same issue, and It looks like dbutils is inaccessible from worker nodes: LINK

  • 4 kudos
2 More Replies
Maverick1
by Valued Contributor II
  • 4986 Views
  • 4 replies
  • 11 kudos

Resolved! Migration from self-managed MLflow to Databricks managed MLflow.

Would it be possible to somehow save the data, metrics of all experiments captured by self-managed mlflow using A/mazon RDS, S3 as backend and then load it to databricks managed mlflow and make it available in the UI? This is required as a part of mi...

  • 4986 Views
  • 4 replies
  • 11 kudos
Latest Reply
User16783853898
Databricks Employee
  • 11 kudos

mlflow-export-import is an open source tool used by both OSS and Databricks users. It is in the process of being a recommended solution for Databricks MLflow migrations. Create a ticket for any issues at: https://github.com/amesar/mlflow-export-impor...

  • 11 kudos
3 More Replies
Marco98
by New Contributor II
  • 1882 Views
  • 1 replies
  • 2 kudos

Pytorch DDP on Databricks

Hello!I am currently trying to use Pytorch Lightning inside Databricks and I am currently using a cluster with 2 gpus. Whenever I try to train my Transformer model with 1 gpu in DP strategy everything works fine, but when I try to use both the 2 gpus...

  • 1882 Views
  • 1 replies
  • 2 kudos
Latest Reply
Chakra03
New Contributor II
  • 2 kudos

Hi @Marco Capusso​ , I am facing the similar issue could you find some fix. It would be great if you share some details around it.

  • 2 kudos
venkat09
by New Contributor III
  • 1833 Views
  • 1 replies
  • 1 kudos

Resolved! Running into an issue while setting up dbx locally?

Followed the documentation and facing issue while running dbx execute on all-purpose/interactive cluster, which is up and running already. Ran this command  dbx execute --cluster-id=XXXXXX --job=dbx-demo-job --no-rebuild --debug. If anyone faced it ...

image
  • 1833 Views
  • 1 replies
  • 1 kudos
Latest Reply
venkat09
New Contributor III
  • 1 kudos

before running package it as wheel before running `dbx execute` fix the issue

  • 1 kudos
Nithin
by New Contributor II
  • 12137 Views
  • 7 replies
  • 1 kudos

How to access databricks feature store outside databricks?

We are building the feature store using databricks API. Few of the machine learning engineers are using Jupyter notebooks. Is it possible to use feature store outside databricks?

  • 12137 Views
  • 7 replies
  • 1 kudos
Latest Reply
datariel
New Contributor II
  • 1 kudos

Hi @Kaniz Fatma​  and @Jose Gonzalez​ ,turning back to the original question, and considering that one of the main benefits of the Feature Store is the removal of the online/offline skew, how could I access to the features from a client application l...

  • 1 kudos
6 More Replies
naveen_marthala
by Contributor
  • 8582 Views
  • 2 replies
  • 3 kudos

Resolved! How to PREVENT mlflow's autologging from logging ALL runs?

I am logging runs from jupyter notebook. the cells which has `mlflow.sklearn.autlog()` behaves as expected. but, the cells which has .fit() method being called on sklearn's estimators are also being logged as runs without explicitly mentioning `mlflo...

  • 8582 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

https://apkmiz.com/showbox-apk-old-latest-version-android/

  • 3 kudos
1 More Replies
Direo
by Contributor II
  • 1618 Views
  • 1 replies
  • 2 kudos

Feature store errors

HiWhen I open feature store, I get an error saying that "Failed to load some job schedules". When I open one of the feature store tables, I get several additional errors:"Failed to laod lates run for some job producers","Failed to laod some job produ...

image
  • 1618 Views
  • 1 replies
  • 2 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 2 kudos

@Direo Direo​ , In that case, I would write to support (in the case of Azure to Microsoft support).

  • 2 kudos
Direo
by Contributor II
  • 7859 Views
  • 1 replies
  • 1 kudos

Resolved! xgboost 1.5.1 gives 'XGBModel' object has no attribute 'enable_categorical' error

Should I pip install xgboost==1.4.2. (the last version it worked) or is there a better way to solve it having in mind that this solution might cause problems later if this version of xgboost is not supported on future python versions.

  • 7859 Views
  • 1 replies
  • 1 kudos
Latest Reply
Direo
Contributor II
  • 1 kudos

Hi, @Kaniz Fatma​. No, I have found a solution. Needed to retrain models using new version of xgboost.

  • 1 kudos
Mr__E
by Contributor II
  • 972 Views
  • 0 replies
  • 0 kudos

Custom AutoML evaluation metric for ranking model

I built a model which is used for ranking and I have a notebook that takes that model to generate rankings and then uses a UDF-based metric to evaluate those rankings. Is there any way that I can have this ranking / UDF be used during the AutoML trai...

  • 972 Views
  • 0 replies
  • 0 kudos
findinpath
by Contributor
  • 5765 Views
  • 6 replies
  • 5 kudos

How to mount s3 bucket in community edition cluster?

I'm using Databricks Community Edition for testing purposes on a OSS project.I'm spinning up the cluster automatically through Databricks Clusters API.The automated tests rely on AWS S3 infrastructure, reason why I need to mount the S3 bucket on the ...

  • 5765 Views
  • 6 replies
  • 5 kudos
Latest Reply
findinpath
Contributor
  • 5 kudos

I haven't found any solution.I'm assuming that currently my only option is the usage of Databricks Enterprise to model scenarios involving the mounting of object storage buckets.

  • 5 kudos
5 More Replies
Wayne
by New Contributor III
  • 1904 Views
  • 2 replies
  • 3 kudos
  • 1904 Views
  • 2 replies
  • 3 kudos
Latest Reply
Wayne
New Contributor III
  • 3 kudos

No error, just seeing the EXPAND DISK in cluster event logs. This is just a regular spark application. I am not sure if the cloud storage matters - a spark application uses it as input and output.

  • 3 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels