- 3077 Views
- 3 replies
- 3 kudos
Can't select cluster when create AutoML experiment
I'm trying to create my experiment using AutoML. I have a running cluster using 12.2 LTS. However, it isn't available as an option for me. How could I solve it? Thank you
- 3077 Views
- 3 replies
- 3 kudos
- 3 kudos
@HAKO411 AutoML needs Databricks Runtime 9.1 ML or above. For time series forecasting, you will need Databricks Runtime 10.0 ML or above. Looks like you are using 12.2 LTS, a non-ML version. Using 12.2 ML LTS should resolve your issue.
- 3 kudos
- 2107 Views
- 2 replies
- 0 kudos
AutoML Runs Failing
After the Data Exploration notebook runs successfully, all AutoML trials fail without providing a source notebook. I have ensured that the training data labels have no null values or any labels with 16 or less occurrences associated with them. I cann...
- 2107 Views
- 2 replies
- 0 kudos
- 0 kudos
@miahopman We understand that you are looking for a better way of troubleshooting in AutoML. We have an internal feature request raised to address precisely the issues you have discussed here.
- 0 kudos
- 6320 Views
- 4 replies
- 1 kudos
Spark connector to mongodb - mongo-spark-connector_2.12:10.1.1
Hello, I´ve added a library to the cluster and it appears in SPARK UI as Added By Userspark://10.139.64.4:43001/jars/addedFile307892533757162075org_mongodb_spark_mongo_spark_connector_2_12_10_1_1-98946.jarAdded By UserI'm trying to connect using the ...
- 6320 Views
- 4 replies
- 1 kudos
- 1 kudos
@DmytroSokhach I think it works if you change mongo to mongodb in the options. and use spark.mongodb.read.connection.uri instead of spark.mongodb.input.uri as @silvadev suggested.
- 1 kudos
- 1148 Views
- 1 replies
- 0 kudos
Classroom Setup Error in LLM Course
Hi All, I encountered this error when running the classroom setup for the LLM Course and would love to know if there's a subscription that Databricks offers to learn and practice these pieces of training.Course name: LLM : Foundation Models from the ...
- 1148 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @gngaPutrBheeshm , Thank you for posting your concern on Community! To expedite your request, please list your concerns on our ticketing portal. Our support staff would be able to act faster on the resolution (our standard resolution time is 24-48...
- 0 kudos
- 1847 Views
- 2 replies
- 0 kudos
Use on-premise MinIO as an artifact store in experiment
Hi. I'm trying to use managed MLflow with our own MinIO as an artifact storage. I can see that there is a description about storage options at landing page and there is an input for artifact store URI when creating empty experiment in databicks works...
- 1847 Views
- 2 replies
- 0 kudos
- 0 kudos
Thanks. I will post there if the feature I asked is doesn't exist.Anyway, the feature I asked about is clearly described on the landing page. I'm looking for documentation for that feature.
- 0 kudos
- 9481 Views
- 8 replies
- 2 kudos
Resolved! Served model creation failed
I have a model registered in unity catalog which works fine and I can load / run and get results returned. I wanted to create a serving endpoint but when I try I get this error.Served model creation failed for served model 'model_name', config versi...
- 9481 Views
- 8 replies
- 2 kudos
- 2 kudos
@kashy Looks like the model is not correctly referenced while loading. You should reference the path of the model till ‘model-best’, which is the top-level directory. loaded_model = mlflow.spacy.load_model("</path/to/your/model/>/model-best")
- 2 kudos
- 6961 Views
- 6 replies
- 3 kudos
How to apply Primary Key constraint in Delta Live Table?
In this blog I can see for dimension and fact tables, the primary key constraint has been applied. Following is the example:-- Store dimensionCREATE OR REPLACE TABLE dim_store( store_id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, business_key ...
- 6961 Views
- 6 replies
- 3 kudos
- 3 kudos
@SRK Please see a copy of this answer on stackoverflow here. You can use DLT Expectations to have this check (see my previous answer if you're using SQL and not Python):@dlt.table(name="table1",)def create_df():schema = T.StructType([T.StructField("i...
- 3 kudos
- 2865 Views
- 3 replies
- 0 kudos
Cannot set experiment in a non-ML runtime
Hello,If we:%pip install mlflow import mlflow mlflow.set_experiment(experiment_name = '/Shared/xx')we get:InvalidConfigurationError: You haven't configured the CLI yet! Please configure by entering `/databricks/python_shell/scripts/db_ipykernel_launc...
- 2865 Views
- 3 replies
- 0 kudos
- 1790 Views
- 2 replies
- 0 kudos
NVIDIA driver update
I want to update the cuda driver for the NVIDIA tesla T4 GPU on the cluster. using the following command%shsudo apt-get --purge remove "*nvidia*"sudo /usr/bin/nvidia-uninstallwget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x8...
- 1790 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @ravi-kolluri_in, Please ensure you've selected the appropriate version for your machine and the Databricks Runtime version being used.
- 0 kudos
- 16344 Views
- 11 replies
- 12 kudos
Resolved! INFORMATION_SCHEMA IS NOT POPULATED WITH TABLE INFORMATION
I have created a metastore and within that metastore i have created multiple schemas and tables underlying it but none of table details is visible from information schema. All the tables are empty.Could you please let me know if I am missing here. Be...
- 16344 Views
- 11 replies
- 12 kudos
- 12 kudos
I noticed this issue is currently caused when you rename a catalog. The contents of <catalog>.information_schema are all views like this:SELECT * FROM system.information_schema.columns WHERE table_catalog = '<catalog>'If you rename the catalog...
- 12 kudos
- 1283 Views
- 1 replies
- 0 kudos
Terraform - Creating Jobs
Hello, Does anyone know how to create a job via Terraform, that automatically overwrites/updates an existing job with the same name?Tried a few different methods but there doesn't seem to be a clean approach. Wondering if anyone has worked this out?I...
- 1283 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @camazg, You can use meta-arguments to overwrite an existing Databricks job with the same name using Terraform. Specifically, you can set the create_before_destroy argument to true, which creates a new resource before destroying the old one.
- 0 kudos
- 1418 Views
- 1 replies
- 0 kudos
databricks.mrm import ModelRiskApi
Hi, I am trying to import/access this module by DBX. However I have issue importing it and I don't find any blog posts/more information about this module beside the public github repo
- 1418 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Anon , If you have installed dbx, but you are still experiencing issues importing the module, make sure that you have activated the Python environment where dbx is installed. To activate a Python environment, navigate to the directory where you...
- 0 kudos
- 3119 Views
- 1 replies
- 0 kudos
MlflowException: Unable to download model artifacts in Databricks while registering model with MLflo
I am attempting to log, register, and deploy a finetuned GPT2 model in Databricks. While I have been able to get my logging code to run, when I try to run my registration code, I get an MlflowException error.Here is my model logging code.mlflow.set_r...
- 3119 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @AChang, I suggest creating a new Databricks cluster and running your code to see if the issue is specific to your current cluster configuration.
- 0 kudos
- 2540 Views
- 2 replies
- 1 kudos
AutoML Trials Failing
Sometimes an AutoML experiment will have all trials fail and I cannot figure out what is causing it. Each individual run reports a validation f1 value but the source notebook is not available so I cannot track down the error. This seems to happen at ...
- 2540 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @miahopman, Did you check the dataset for errors, missing values or other anomalies affecting the AutoML performance?
- 1 kudos
- 4367 Views
- 2 replies
- 2 kudos
MLFlow model loading taking long time and "model serving" failing during init
I am trying to load a simple Minmaxscaler model that was logged as a run through spark's ML Pipeline api for reuse. On average it takes 40+ seconds just to load the model with the following example: This is fine and the model transforms my data corre...
- 4367 Views
- 2 replies
- 2 kudos
- 2 kudos
Hello,Any solutions found for this issue?I'm serving up a large number of models at a time, but since we converted to PySpark (due to our data demands), the mlflow.spark.load_model() is taking hours.Part of the reason to switch to spark was to help w...
- 2 kudos
