- 839 Views
- 2 replies
- 0 kudos
Error on Workflow: Failure to initialize configuration for storage account
I have set up a workflow with a sequence of jobs. Each job run fine in an interactive mode, that is, run the notebook directly. However, when I tried to run the workflow, it got error on a step which uses a function from a Repo. the error says "Failu...
- 839 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Tingting ,It seems that when you run notebook interactively, your personal credentials are used to access ADLS.When the workflow job is run, Databricks uses different context. Could you share whether your job is accessing some storage account, an...
- 0 kudos
- 1020 Views
- 1 replies
- 0 kudos
Resolved! Mlflow not saving flavor correctly
Hello!Im trying to save my model with mlflow in databricks, it is a xgboost model, when I save it using code it saves with a sklearn flavor and not saves other parameters, also I'm using kedro with kedro-mlflow plugin.def log_metrics_and_model(model,...
- 1020 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello!It was the magic of all porpoise clusters, just restart the cluster and done x.x
- 0 kudos
- 1453 Views
- 0 replies
- 0 kudos
Model serving with custom pip index URL
An mlflow model was logged with a custom pip requirements file which contains package versions (mlflow==2.11.3), as well as a custom --index-url. However model serving during the "Initializing model enviroment" step tries to pip install mlflow==2.2.2...
- 1453 Views
- 0 replies
- 0 kudos
- 1242 Views
- 0 replies
- 0 kudos
Error to create an endpoint of databricks with 2 primary keys online table
I have a delta table that has a primary key conformed by 2 fields (accountId,ruleModelVersionDesc) and I have also created an online table that has the same primary key, but when I create a feature spec to create an endpoint I get the following error...
- 1242 Views
- 0 replies
- 0 kudos
- 1271 Views
- 0 replies
- 0 kudos
Machine Learning Model Deployment on Databricks with Unity Catalog
Hi everyone! I found it would be helpful to document and share my experiences navigating model deployment on Databricks with the recent changes to deploy models to Unity Catalog instead of the Workspace Model Registry. https://medium.com/p/7d04e85395...
- 1271 Views
- 0 replies
- 0 kudos
- 1739 Views
- 5 replies
- 2 kudos
Resolved! Uninstall whl file from databricks cluster via CLI
Hello, we have a need to uninstall older versions of whl files from personal cluster via databricks CLI - could you please provide the exact command to be used here. We tried with many found on the documentations but none of them worked to do the act...
- 1739 Views
- 5 replies
- 2 kudos
- 2 kudos
Hi @Visakh_Vijayan ,Did you try to use databricks libraries uninstall? It's exactly crafted for this purposedatabricks libraries uninstall --json YOUR_JSON_WITH_REQUEST_BODYAlso, when you uninstall a library from a cluster, the library is removed onl...
- 2 kudos
- 4231 Views
- 2 replies
- 0 kudos
problem switching profile when using databricks cli
HiI have installed databricks CLI and have created some different profiles, and havnt had any problems until now. When i try to use a specific profile with my commands using the --profile flag, fx "databricks secrets list-scopes --profile prod" i enc...
- 4231 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @NielsMH ,By default, the Databricks CLI looks for the .databrickscfg file in your ~ (your user home) folder on Unix. You can try to delete this file and run configuration process again.You can also use describe command to check what credentials a...
- 0 kudos
- 1268 Views
- 0 replies
- 0 kudos
Can't query Legacy Serving Endpoint
Hi,I was able to deploy an endpoint using legacy serving (It's the only option we have to deploy endpoints in DB). Now I am having trouble querying the endpoint itself. When I try to query it I get the following error: Here is the code I am using ...
- 1268 Views
- 0 replies
- 0 kudos
- 715 Views
- 2 replies
- 0 kudos
machine learning compute cluster?
can the community edition be allowed to create a machine learning compute cluster?
- 715 Views
- 2 replies
- 0 kudos
- 0 kudos
No, most of the platforms, including services like Azure and Databricks, are not able to allow the creation of a machine learning compute cluster on this community edition. Most of these editions allow only basic features and resources enough to expl...
- 0 kudos
- 1754 Views
- 0 replies
- 1 kudos
Multi-tenant recommendation system (Machine learning)
Hello,I am looking to build a multi-tenant machine learning recommender system in Azure Databricks. The idea is to have a single shared model, where each tenant can use the same model to train on their own unique dataset. Essentially, while the model...
- 1754 Views
- 0 replies
- 1 kudos
- 1321 Views
- 0 replies
- 0 kudos
Determine exact location of MLflow model tracking and model registry files and the Backend Stores
I would like to determine the exact location of:1. MLflow model tracking files2. Model registry files (with Workspace Model Registry)as according to the documentation it is mentioned that: "All methods copy the model into a secure location managed by...
- 1321 Views
- 0 replies
- 0 kudos
- 3407 Views
- 3 replies
- 0 kudos
Error installing datasets needed for LLM course
I signed up for this course via Databricks Academy : LLMs: Application through Production However I am getting this error when trying to download the needed datasets for the course:Installing datasets:| from "wasbs://courseware@dbacademy.blob.core.wi...
- 3407 Views
- 3 replies
- 0 kudos
- 0 kudos
You would need to install the python library. You can either:1) Run %pip install datasets2) Put it as part of the PyPi packages to load in your cluster This should solve your issue
- 0 kudos
- 1363 Views
- 2 replies
- 0 kudos
AutoML split with dt column not working properly
I am using AutoML and want to split my data to train/validation and test using a dt column (one date for train one different date for validation and a third date for test. The problem that the autoML fails, there are only training metrics (no valiat...
- 1363 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello! Did you try specify a column name as manual split column? Then you can fully control which rows are in train / validate / test splits: https://docs.databricks.com/en/machine-learning/automl/automl-data-preparation.html#split-data-for-regressi...
- 0 kudos
- 1320 Views
- 0 replies
- 0 kudos
Why is spark mllib is not encouraged on the platform?/Why is ML dependent on .toPandas() on dbricks?
I'm new to Spark,Databricks and am surprised about how the Databricks tutorials for ML are using pandas DF > Spark DF. Of the tutorials I've seen, most data processing is done in a distributed manner but then its just cast to a pandas dataframe. From...
- 1320 Views
- 0 replies
- 0 kudos
- 1385 Views
- 0 replies
- 0 kudos
FeatureEngineeringClient and Unity Catalog
When testing this code ( fe.score_batch( df=dataset.drop("Target").limit(10), model_uri=f"models:/{model_name}/{mv.version}", ) .select("prediction") .limit(10) .display() ) I get the error: “MlflowException: The...
- 1385 Views
- 0 replies
- 0 kudos
