cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Science & Machine Learning

Forum Posts

lchicoma
by New Contributor
  • 235 Views
  • 0 replies
  • 0 kudos

Error to create an endpoint of databricks with 2 primary keys online table

I have a delta table that has a primary key conformed by 2 fields (accountId,ruleModelVersionDesc) and I have also created an online table that has the same primary key, but when I create a feature spec to create an endpoint I get the following error...

Machine Learning
enpoints
featurespec
fetureserving
MachineLearning
onlinetabl
  • 235 Views
  • 0 replies
  • 0 kudos
ac10
by New Contributor II
  • 229 Views
  • 0 replies
  • 0 kudos

Machine Learning Model Deployment on Databricks with Unity Catalog

Hi everyone! I found it would be helpful to document and share my experiences navigating model deployment on Databricks with the recent changes to deploy models to Unity Catalog instead of the Workspace Model Registry. https://medium.com/p/7d04e85395...

  • 229 Views
  • 0 replies
  • 0 kudos
Djay101
by New Contributor
  • 342 Views
  • 0 replies
  • 0 kudos

How do we log without a dbfs for MLFlow models.

Hi Databricks Team,We are planning a UC Migration for a customer who currently has around 500 experiments, each with multiple runs. These experiments are registered and MLflow is logging to DBFS locations. However, we have not found any documentation...

  • 342 Views
  • 0 replies
  • 0 kudos
Visakh_Vijayan
by New Contributor II
  • 758 Views
  • 5 replies
  • 2 kudos

Resolved! Uninstall whl file from databricks cluster via CLI

Hello, we have a need to uninstall older versions of whl files from personal cluster via databricks CLI - could you please provide the exact command to be used here. We tried with many found on the documentations but none of them worked to do the act...

  • 758 Views
  • 5 replies
  • 2 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 2 kudos

Hi @Visakh_Vijayan ,Did you try to use databricks libraries uninstall? It's exactly crafted for this purposedatabricks libraries uninstall --json YOUR_JSON_WITH_REQUEST_BODYAlso, when you uninstall a library from a cluster, the library is removed onl...

  • 2 kudos
4 More Replies
NielsMH
by New Contributor III
  • 850 Views
  • 2 replies
  • 0 kudos

problem switching profile when using databricks cli

HiI have installed databricks CLI and have created some different profiles, and havnt had any problems until now. When i try to use a specific profile with my commands using the --profile flag, fx "databricks secrets list-scopes --profile prod" i enc...

  • 850 Views
  • 2 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @NielsMH ,By default, the Databricks CLI looks for the .databrickscfg file in your ~ (your user home) folder on Unix. You can try to delete this file and run configuration process again.You can also use describe command to check what credentials a...

  • 0 kudos
1 More Replies
semsim
by Contributor
  • 209 Views
  • 0 replies
  • 0 kudos

Can't query Legacy Serving Endpoint

Hi,I was able to deploy an endpoint using legacy serving (It's the only option we have to deploy endpoints in DB). Now I am having trouble querying the endpoint itself. When I try to query it I get the following error:    Here is the code I am using ...

semsim_0-1726245119742.png
  • 209 Views
  • 0 replies
  • 0 kudos
dzhou
by New Contributor
  • 386 Views
  • 2 replies
  • 0 kudos

machine learning compute cluster?

can the community edition be allowed to create a machine learning compute cluster?

  • 386 Views
  • 2 replies
  • 0 kudos
Latest Reply
michael569gardn
New Contributor III
  • 0 kudos

No, most of the platforms, including services like Azure and Databricks, are not able to allow the creation of a machine learning compute cluster on this community edition. Most of these editions allow only basic features and resources enough to expl...

  • 0 kudos
1 More Replies
dcunningham1
by New Contributor
  • 370 Views
  • 0 replies
  • 0 kudos

Possible to use `params` argument of `mlflow.pyfunc.PythonModel` deployed to Databricks endpoint?

I'm deploying a custom model using the `mlflow.pyfunc.PythonModel` class as described here. My model uses the `params` argument in the `predict` method to allow the user to choose some aspects of the model at inference time. For example:class CustomM...

  • 370 Views
  • 0 replies
  • 0 kudos
Kasen
by New Contributor III
  • 598 Views
  • 0 replies
  • 1 kudos

Multi-tenant recommendation system (Machine learning)

Hello,I am looking to build a multi-tenant machine learning recommender system in Azure Databricks. The idea is to have a single shared model, where each tenant can use the same model to train on their own unique dataset. Essentially, while the model...

Machine Learning
machine learning
multi-tenant
recommendation
  • 598 Views
  • 0 replies
  • 1 kudos
NielsMH
by New Contributor III
  • 1354 Views
  • 3 replies
  • 3 kudos

Resolved! Using variables with Databricks Asset Bundles not working

Hi EveryoneI am using the default Databricks Asset Bundle. All is good except i can get stage dependt variables to work.i have a bundle config file that looks like the below, where i have defined variables for each stage.bundle: name: xxxxx includ...

  • 1354 Views
  • 3 replies
  • 3 kudos
Latest Reply
NielsMH
New Contributor III
  • 3 kudos

Hi Slash. That was indeed the problem. Thanks!

  • 3 kudos
2 More Replies
ScyLukb
by New Contributor
  • 272 Views
  • 0 replies
  • 0 kudos

Determine exact location of MLflow model tracking and model registry files and the Backend Stores

I would like to determine the exact location of:1. MLflow model tracking files2. Model registry files (with Workspace Model Registry)as according to the documentation it is mentioned that: "All methods copy the model into a secure location managed by...

  • 272 Views
  • 0 replies
  • 0 kudos
163050
by New Contributor II
  • 2930 Views
  • 3 replies
  • 0 kudos

Error installing datasets needed for LLM course

I signed up for this course via Databricks Academy : LLMs: Application through Production However I am getting this error when trying to download the needed datasets for the course:Installing datasets:| from "wasbs://courseware@dbacademy.blob.core.wi...

163050_0-1696757589160.png
  • 2930 Views
  • 3 replies
  • 0 kudos
Latest Reply
david_for_db
New Contributor II
  • 0 kudos

You would need to install the python library. You can either:1) Run %pip install datasets2) Put it as part of the PyPi packages to load in your cluster This should solve your issue

  • 0 kudos
2 More Replies
Noura_azza
by New Contributor II
  • 1015 Views
  • 2 replies
  • 0 kudos

AutoML split with dt column not working properly

I am using AutoML and want to split my data to train/validation and test  using a dt column (one date for train one different date for validation and a third date for test. The problem that the autoML fails, there are only training metrics (no valiat...

  • 1015 Views
  • 2 replies
  • 0 kudos
Latest Reply
maggiewang
Databricks Employee
  • 0 kudos

Hello! Did you try specify a column name as manual split column?  Then you can fully control which rows are in train / validate / test splits: https://docs.databricks.com/en/machine-learning/automl/automl-data-preparation.html#split-data-for-regressi...

  • 0 kudos
1 More Replies
stochastic
by New Contributor
  • 264 Views
  • 0 replies
  • 0 kudos

Why is spark mllib is not encouraged on the platform?/Why is ML dependent on .toPandas() on dbricks?

I'm new to Spark,Databricks and am surprised about how the Databricks tutorials for ML are using pandas DF > Spark DF. Of the tutorials I've seen, most data processing is done in a distributed manner but then its just cast to a pandas dataframe. From...

  • 264 Views
  • 0 replies
  • 0 kudos
Kjetil
by New Contributor III
  • 288 Views
  • 0 replies
  • 0 kudos

FeatureEngineeringClient and Unity Catalog

When testing this code  ( fe.score_batch( df=dataset.drop("Target").limit(10), model_uri=f"models:/{model_name}/{mv.version}", ) .select("prediction") .limit(10) .display() )  I get the error: â€œMlflowException: The...

  • 288 Views
  • 0 replies
  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels