- 2722 Views
- 1 replies
- 1 kudos
Resolved! Running into an issue while setting up dbx locally?
Followed the documentation and facing issue while running dbx execute on all-purpose/interactive cluster, which is up and running already. Ran this command dbx execute --cluster-id=XXXXXX --job=dbx-demo-job --no-rebuild --debug. If anyone faced it ...
- 2722 Views
- 1 replies
- 1 kudos
- 1 kudos
before running package it as wheel before running `dbx execute` fix the issue
- 1 kudos
- 14932 Views
- 7 replies
- 1 kudos
How to access databricks feature store outside databricks?
We are building the feature store using databricks API. Few of the machine learning engineers are using Jupyter notebooks. Is it possible to use feature store outside databricks?
- 14932 Views
- 7 replies
- 1 kudos
- 1 kudos
Hi @Kaniz Fatma​ and @Jose Gonzalez​ ,turning back to the original question, and considering that one of the main benefits of the Feature Store is the removal of the online/offline skew, how could I access to the features from a client application l...
- 1 kudos
- 2408 Views
- 1 replies
- 2 kudos
Feature store errors
HiWhen I open feature store, I get an error saying that "Failed to load some job schedules". When I open one of the feature store tables, I get several additional errors:"Failed to laod lates run for some job producers","Failed to laod some job produ...
- 2408 Views
- 1 replies
- 2 kudos
- 2 kudos
@Direo Direo​ , In that case, I would write to support (in the case of Azure to Microsoft support).
- 2 kudos
- 10581 Views
- 1 replies
- 1 kudos
Resolved! xgboost 1.5.1 gives 'XGBModel' object has no attribute 'enable_categorical' error
Should I pip install xgboost==1.4.2. (the last version it worked) or is there a better way to solve it having in mind that this solution might cause problems later if this version of xgboost is not supported on future python versions.
- 10581 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, @Kaniz Fatma​. No, I have found a solution. Needed to retrain models using new version of xgboost.
- 1 kudos
- 1404 Views
- 0 replies
- 0 kudos
Custom AutoML evaluation metric for ranking model
I built a model which is used for ranking and I have a notebook that takes that model to generate rankings and then uses a UDF-based metric to evaluate those rankings. Is there any way that I can have this ranking / UDF be used during the AutoML trai...
- 1404 Views
- 0 replies
- 0 kudos
- 7676 Views
- 6 replies
- 5 kudos
How to mount s3 bucket in community edition cluster?
I'm using Databricks Community Edition for testing purposes on a OSS project.I'm spinning up the cluster automatically through Databricks Clusters API.The automated tests rely on AWS S3 infrastructure, reason why I need to mount the S3 bucket on the ...
- 7676 Views
- 6 replies
- 5 kudos
- 5 kudos
I haven't found any solution.I'm assuming that currently my only option is the usage of Databricks Enterprise to model scenarios involving the mounting of object storage buckets.
- 5 kudos
- 2735 Views
- 2 replies
- 3 kudos
- 2735 Views
- 2 replies
- 3 kudos
- 3 kudos
No error, just seeing the EXPAND DISK in cluster event logs. This is just a regular spark application. I am not sure if the cloud storage matters - a spark application uses it as input and output.
- 3 kudos
- 3157 Views
- 1 replies
- 0 kudos
Databricks online store - Login to Azure SQL Database with Service Principal
I want to use Databricks Online Store with Azure SQL Database, however I am unable to autenthicate through Databricks Feature Store API. I need to use Service Principal credentials.I tried using Application ID as username and Secret as password, but ...
- 3157 Views
- 1 replies
- 0 kudos
- 0 kudos
no one?
- 0 kudos
- 5404 Views
- 2 replies
- 2 kudos
why does the client need to have git installed for auto-logging to an mlflow server running in "--serve-artifacts" mode?
I have an mlflow server with `--serve-artifacts` and with postgres as `--backend-store-uri`. The machine(container with base image python:3.9-bullseye) running the server has git installed which is available on path. I am logging from jupyter-noteboo...
- 5404 Views
- 2 replies
- 2 kudos
- 25696 Views
- 4 replies
- 1 kudos
Resolved! Set default "spark.driver.maxResultSize" from the notebook
Hello,I would like to set the default "spark.driver.maxResultSize" from the notebook on my cluster. I know I can do that in the cluster settings, but is there a way to set it by code?I also know how to do it when I start a spark session, but in my ca...
- 25696 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @Maximilian Hansinger​ Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark the answer as best? If not, please tell us so we can help you.Thanks!
- 1 kudos
- 2238 Views
- 0 replies
- 0 kudos
Pymc3 on Databricks: Progress bar
Hello everybody..I am trying to run pymc3 models on Databricks (runtime 9.1) and when I start the sampling process, the progress bar is not showing. It is a bit annoying since this way I do not have any information on when the process is going to end...
- 2238 Views
- 0 replies
- 0 kudos
- 5503 Views
- 4 replies
- 2 kudos
Resolved! Cluster setup for ML work for Pandas in Spark, and vanilla Python.
My setup:Worker type: Standard_D32d_v4, 128 GB Memory, 32 Cores, Min Workers: 2, Max Workers: 8Driver type: Standard_D32ds_v4, 128 GB Memory, 32 CoresDatabricks Runtime Version: 10.2 ML (includes Apache Spark 3.2.0, Scala 2.12)I ran a snowflake quer...
- 5503 Views
- 4 replies
- 2 kudos
- 2 kudos
Hey there @Vivek Ranjan​ Checking in. If Joseph's answer helped, would you let us know and mark the answer as best? It would be really helpful for the other members to find the solution more quickly.Thanks!
- 2 kudos
- 5320 Views
- 1 replies
- 2 kudos
Resolved! How to deploy or create mlflow model as docker image with REST api endpoint within databricks?
Is it possible to create mlflow model as a docker image with REST api endpoint and use it for inferencing within databricks or hosting the image in azure container instances?
- 5320 Views
- 1 replies
- 2 kudos
- 2 kudos
@Vijeth Moudgalya​ , Hey there, we are definitely interested in making model serving easier and simpler on Databricks. There are some useful product features coming down the line - contact me at bilal dot aslam at databricks dot com if you are intere...
- 2 kudos
- 12353 Views
- 4 replies
- 2 kudos
Resolved! In Databricks, the Python kafka consumer app in notebook to Confluent Cloud having the issue captured in the Body of question: SASL/PLAIN authentication being used
kafkashaded.org.apache.kafka.common.KafkaException: Failed to construct kafka consumer at kafkashaded.org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:823) at kafkashaded.org.apache.kafka.clients.consumer.KafkaConsumer.<init>...
- 12353 Views
- 4 replies
- 2 kudos
- 2 kudos
@Kaniz Fatma​ I am having the same issue.%python import pyspark.sql.functions as fn from pyspark.sql.types import StringType binary_to_string = fn.udf(lambda x: str(int.from_bytes(x, byteorder='big')), StringType()) df = spark.readStream.format("...
- 2 kudos
- 4490 Views
- 3 replies
- 0 kudos
Community Edition - MLflow RestException: PERMISSION_DENIED: Model Registry is not enabled for organization
I am trying the databricks community edition . When i try to register my model it is throwing following exception.RestException: PERMISSION_DENIED: Model Registry is not enabled for organizationHow to enable the model registry ... I have tried in set...
- 4490 Views
- 3 replies
- 0 kudos
