- 762 Views
- 2 replies
- 1 kudos
Endpoint creation without scale-to-zero
Hi, I've got a question about deploying an endpoint for Llama 3.1 8b. The following code should create the endpoint without scale-to-zero. The endpoint is being created, but with scale-to-zero, although scale_to_zero_enabled is set to False. Instead ...
- 762 Views
- 2 replies
- 1 kudos
- 1 kudos
Thanks for the reply @Walter_C. This didn't quite work, since it used a CPU and didn't consider the max_provisioned_throughput, but I finally got it to work like this: from mlflow.deployments import get_deploy_client client = get_deploy_client("data...
- 1 kudos
- 6885 Views
- 5 replies
- 2 kudos
Issue with Multi-column In predicates are not supported in the DELETE condition.
I'm trying to delete rows from a table with the same date or id as records in another table. I'm using the below query and get the error 'Multi-column In predicates are not supported in the DELETE condition'. delete from cost_model.cm_dispatch_consol...
- 6885 Views
- 5 replies
- 2 kudos
- 2 kudos
I seem to get this error on some DeltaTables and not others:df.createOrReplaceTempView("channels_to_delete") spark.sql(""" delete from lake.something.earnings where TenantId = :tenantId and ChannelId = in ( select ChannelId ...
- 2 kudos
- 2118 Views
- 3 replies
- 1 kudos
Resolved! Extracting Topics From Text Data Using PySpark
Hi EveryoneI tried to follow the same steps in Topic from Text on similar data as example. However, when I tri to fit the model with data I get this error.IllegalArgumentException: requirement failed: Column features must be of type equal to one of t...
- 2118 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @amirA ,The LDA model expects the features column to be of type Vector from the pyspark.ml.linalg module, specifically either a SparseVector or DenseVector, whereas you have provided Row type.You need to convert your Row object to SparseVector.Che...
- 1 kudos
- 4362 Views
- 15 replies
- 2 kudos
Serving Endpoint Container Image Creation Fails
Hello, I trained a model using MLFlow, and saved the model as an artifact. I can load the model from a notebook and it works as expected (i.e. I can load the model using its URI).However, when I want to deploy it using Databricks endpoints, container...
- 4362 Views
- 15 replies
- 2 kudos
- 2 kudos
@ivan_calvo The problem still exists. Surely there has to be some other option than downgrading the ML cluster to DBR 14.3 LTS ML?
- 2 kudos
- 676 Views
- 2 replies
- 0 kudos
I want to develop an automated lead allocation system to prospect sales representatives.
I want to develop an automated lead allocation system to prospect sales representatives. Please suggest a suitable solution also any links if available.
- 676 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi jamesl,My use case is related to match the prospect sales agent for the customer entering retail store, when a customer enters a store based on the inputs provided and checking on if the customer is existing or new customer, I want to create a rea...
- 0 kudos
- 2881 Views
- 6 replies
- 4 kudos
- 2881 Views
- 6 replies
- 4 kudos
- 4 kudos
There could be multiple reasone why you're getting this error @avishkarborkar . If the course you're following requires Unity Catalog, first you need to check if you have a premium workspace. Next you need to make sure that your workspace is enabled ...
- 4 kudos
- 2033 Views
- 1 replies
- 0 kudos
Unable to Check Experiment Existence with path starting with /Workspace/ Directory in Databricks Pla
https://github.com/mlflow/mlflow/issues/11077 In Databricks, when attempting to set an experiment with an experiment_name specified as an absolute path from /Workspace/Shared/mlflow_experiment/<experiment_name>, the mlflow.set_experiment() function ...
- 2033 Views
- 1 replies
- 0 kudos
- 0 kudos
Before setting the experiment, use mlflow.get_experiment_by_name() to check if the experiment already exists. If it does, you can set the experiment without attempting to create it again.
- 0 kudos
- 504 Views
- 1 replies
- 0 kudos
What is the best to way to not deploy/run a workflow in production?
I am building and MLOps architecture.I do not want to deploy the training workflow to prod. My first approach was to selectively not deploy the workflow to prod, but this does not seem to be possible as in this thread:https://community.databricks.com...
- 504 Views
- 1 replies
- 0 kudos
- 0 kudos
Target Override Feature: You can use the target override feature to specify different configurations for different environments. However, this does not provide a direct way to exclude specific job resources.Environment-Specific Folders: Another app...
- 0 kudos
- 741 Views
- 1 replies
- 0 kudos
request for exam certification voucher
Hi , I've completed the course Machine Learning with Databricks ! Looking forward to learn more .
- 741 Views
- 1 replies
- 0 kudos
- 0 kudos
- 0 kudos
- 4618 Views
- 8 replies
- 0 kudos
One-hot encoding of strong cardinality features failing, causes downstream issues
Hi Databricks support,I'm training an ML model using mlflow on DBR 13.3 LTS ML, Spark 3.4.1 using databricks.automl_runtime 0.2.17 and databricks.automl 1.20.3, with shap 0.45.1. My training data has two float-type columns with three or fewer unique ...
- 4618 Views
- 8 replies
- 0 kudos
- 0 kudos
Hi @rtreves , sorry I was not able to investigate on the above. Not sure if you would be able to create a support ticket with Databricks as it may be an involved effort to review the code. I do have a suggestion, instead of relying on the automatic ...
- 0 kudos
- 1292 Views
- 1 replies
- 1 kudos
Resolved! Serving model with custom scoring script to a real-time endpoint
Hi, new to databricks here and wasn't able to find relevant info in the documentation.Is it not possible to serve a model with a custom scoring script to an online endpoint on databricks to customise inference ? the customisation is related to incomi...
- 1292 Views
- 1 replies
- 1 kudos
- 1 kudos
If I'm understanding, all you really want to do is have a pre/post - process function running with your model, is that correct? If so, you can do this by using the MLflow pyfunc model. Something like they do here:https://docs.databricks.com/en/machi...
- 1 kudos
- 1233 Views
- 0 replies
- 1 kudos
Table-Model Lineage for models without online Feature Lookups
Hi community,I am looking for the recommended way to achieve table-model lineage in Unity Catalog for models that don't use Feature Lookups but only offline features. When I use FeatureEngineeringClient.create_training_set with feature_lookups + mlfl...
- 1233 Views
- 0 replies
- 1 kudos
- 1171 Views
- 3 replies
- 0 kudos
Consequences of Not Using write_table with Feature Engineering Client and INSERT OVERWRITE
Hello Databricks Community,I am currently using the Feature Engineering client and have a few questions about best practices for writing to Feature Store Tables.I would like to know more about not using the write_table method directly from the featur...
- 1171 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @zed,How are you doing? As per my understanding, Consider using the write_table method from the Feature Engineering client to ensure that all Feature Store functionality is properly leveraged, such as cataloging, lineage tracking, and handling upd...
- 0 kudos
- 1008 Views
- 0 replies
- 0 kudos
Hyperopt (15.4 LTS ML) ignores autologger settings
I use ML Flow Experiment to store models once they leave very early tests and development. I switched lately to 15.4 LTS ML and was hit by unhinged Hyperopt behavior:it was creating Experiment logs ignoring i) autologger is off on the workspace level...
- 1008 Views
- 0 replies
- 0 kudos
- 1015 Views
- 0 replies
- 2 kudos
Bug: MLflow recipe
I'm not sure whether this is the right place, but we've encountered a bug in the datasets.py(https://github.com/mlflow/mlflow/blob/master/mlflow/recipes/steps/ingest/datasets.py.). Anyone using recipes beware of forementioned.def _convert_spark_df_to...
- 1015 Views
- 0 replies
- 2 kudos
