cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Science & Machine Learning

Forum Posts

Raghav1
by New Contributor II
  • 485 Views
  • 0 replies
  • 0 kudos

I have created a key in Azure Key Vault to store my secrets in it. In order to use it securely in Azure DataBricks, have created the secret scope and ...

I have created a key in Azure Key Vault to store my secrets in it. In order to use it securely in Azure DataBricks, have created the secret scope and configured the Azure Key Vault properties. Out of curiosity, just wanted to check whether my key is ...

databricks issue
  • 485 Views
  • 0 replies
  • 0 kudos
Dan_Z
by Honored Contributor
  • 407 Views
  • 0 replies
  • 0 kudos

spark.apache.org

mapInPandas is one of the most powerful Spark functions. It uses an arrow-like in-memory data structure to split up Spark Data Frames into chunks and feeding them to a function that takes a Pandas DF as input and output. Check it out here:https://spa...

  • 407 Views
  • 0 replies
  • 0 kudos
User16835756816
by Valued Contributor
  • 743 Views
  • 0 replies
  • 6 kudos

Ready to get hands-on? Explore the collaborative notebook environment: This gallery showcases some of the possibilities through Notebooks focused on ...

Ready to get hands-on? Explore the collaborative notebook environment: This gallery showcases some of the possibilities through Notebooks focused on technologies and use cases which can easily be imported into your own Databricks environment or the f...

  • 743 Views
  • 0 replies
  • 6 kudos
Joseph_B
by New Contributor III
  • 908 Views
  • 0 replies
  • 1 kudos

mlflow.org

2021-09 webinar: Automating the ML Lifecycle With Databricks Machine Learning (Post 2 of 2)Thank you to everyone who joined! You can access the on-demand recording here and the code in this Github repo.We're sharing a subset of the questions asked an...

  • 908 Views
  • 0 replies
  • 1 kudos
Joseph_B
by New Contributor III
  • 646 Views
  • 0 replies
  • 1 kudos

docs.databricks.com

2021-09 webinar: Automating the ML Lifecycle With Databricks Machine Learning (post 1 of 2)Thank you to everyone who joined the Automating the ML Lifecycle With Databricks Machine Learning webinar! You can access the on-demand recording here and the ...

  • 646 Views
  • 0 replies
  • 1 kudos
Joseph_B
by New Contributor III
  • 818 Views
  • 1 replies
  • 0 kudos

What does it mean if an MLflow run is "UNFINISHED?"

The run has a model logged to it already, but the "Status" field says "UNFINISHED."

  • 818 Views
  • 1 replies
  • 0 kudos
Latest Reply
Joseph_B
New Contributor III
  • 0 kudos

The MLflow run was probably created either (a) via notebook autologging or (b) via a call to `mlflow.start_run()`. With (a), when the notebook first logs something to MLflow, it starts a run. But if the notebook is still active and attached to a clu...

  • 0 kudos
MCosta
by New Contributor III
  • 1127 Views
  • 2 replies
  • 1 kudos

Delta Lake as source of images to train a classification model on a local computer

Hi Folks, I'm evaluating Delta Lake to store image / data version control to be used to train models. I looked at a session explaining how to do this and also using MLflow to manage training (https://databricks.com/session_na21/image-processing-on-d...

  • 1127 Views
  • 2 replies
  • 1 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 1 kudos

I can think of 3 ways for doing this:using the web UI (the create table option or upload data into DBFS)using databricks-connect, which bridges your local machine with the remote databricks clustersusing the databricks-cli to copy local files to dbfs...

  • 1 kudos
1 More Replies
User16826992666
by Valued Contributor
  • 1164 Views
  • 1 replies
  • 0 kudos

MLflow not logging metrics

I have run a few MLflow experiments and I can see them in the experiment history, but none of the metrics have been logged along with them. I thought this was supposed to be automatically included. Any idea why they wouldn't be showing up?

  • 1164 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @ trevor.bishop! My name is Kaniz, and I'm the technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers on the Forum have an answer to your question first. Or else I will follow up shortly with a response.

  • 0 kudos
User15787040559
by New Contributor III
  • 1910 Views
  • 3 replies
  • 0 kudos

Is there a way to add an instance profile to a Databricks group using REST API?

Please seehttps://docs.databricks.com/dev-tools/api/latest/scim/scim-groups.htmlAn examplecurl -X PATCH -n 'https://test-fw-us-1.cloud.databricks.com/api/2.0/preview/scim/v2/Groups/4062877510650442' -d \ "{ \"schemas\":[\"urn:ietf:params:scim:api:mes...

  • 1910 Views
  • 3 replies
  • 0 kudos
Latest Reply
Cary
New Contributor III
  • 0 kudos

Without knowing all that you are trying to do, the answer is yes, with the Instance Profile API. https://docs.databricks.com/dev-tools/api/latest/instance-profiles.html. You might also check out the SCIM APIs to associate the Instance Profile to a g...

  • 0 kudos
2 More Replies
javierxperez60
by New Contributor II
  • 817 Views
  • 1 replies
  • 0 kudos

Runtime 8.4 ML still showing TF 2.3

I updated my cluster runtime to use 8.4 ML, I did a !pip list and saw that tensorflow was 2.3. What should I do to resolve this? Not sure if I should just do a pip install Tensorflow==2.5.0

  • 817 Views
  • 1 replies
  • 0 kudos
Latest Reply
javierxperez60
New Contributor II
  • 0 kudos

If the answer is the latter, do i need to install the supported versions of cudnn and cuda?

  • 0 kudos
Anonymous
by Not applicable
  • 780 Views
  • 1 replies
  • 0 kudos
  • 780 Views
  • 1 replies
  • 0 kudos
Latest Reply
KevinC_
New Contributor II
  • 0 kudos

I think I need a little more context here on what you're trying to achieve. If you're generally interested in schema evolution, this post talks about feature store: https://databricks.com/blog/2021/05/27/databricks-announces-the-first-feature-store-i...

  • 0 kudos
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels