cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Machine Learning
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Solide
by • New Contributor
  • 5327 Views
  • 8 replies
  • 2 kudos

Databricks runtime version Error

Hello,I'm following courses on the Databricks academy and using for that purpose the Databricks Community edition using a runtime 12.2 LTS (includes Apache Spark 3.3.2, Scala 2.12) and I believe it can't be changedI'm following the Data engineering c...

Machine Learning
Reply
  • 5327 Views
  • 8 replies
  • 2 kudos
View Replies
Latest Reply
V2dha
New Contributor II
  • 2 kudos

I was facing the same error. This could be resolved by adding the version that you are currently working with in the config function present in '_common' notebook in the "Includes' folder. (This was the case of my folder structure that I downloaded f...

  • 2 kudos
7 More Replies
anvil
by • New Contributor II
  • 1527 Views
  • 3 replies
  • 4 kudos

Are UDFs necessary for applying models from ML libraries at scale ?

Hello,I recently finished the "scalable machine learning with apache spark" course and saw that SKLearn models could be applied faster in a distributed manner when used in pandas UDFs or with mapInPandas() method. Spark MLlib models don't need this k...

Machine Learning
Reply
  • 1527 Views
  • 3 replies
  • 4 kudos
View Replies
Latest Reply
Manoj12421
Valued Contributor II
  • 4 kudos

MlLib is in the maintenance model and udf is not used by creating model in most cases

  • 4 kudos
2 More Replies
lukaszl
by • New Contributor III
  • 11082 Views
  • 6 replies
  • 7 kudos

Resolved! Access the environment variable from the custom container base cluster

Hi Databricks Community, I want to set environment variables for all clusters in my workspace. The goal is to the have environment variable, available in all notebooks executed on the cluster.The environment variable is generated in global init scrip...

Machine Learning
Reply
  • 11082 Views
  • 6 replies
  • 7 kudos
View Replies
Latest Reply
grazie
Contributor
  • 7 kudos

Thanks @Lukasz Lu​ - that worked for me as well. When I used the following script:#!/bin/bash echo MY_TEST_VAR=value1 | tee -a /etc/environment >> /databricks/spark/conf/spark-env.shfor non-docker clusters, MY_TEST_VAR shows up twice in ` /databrick...

  • 7 kudos
5 More Replies
THIAM_HUATTAN
by • Valued Contributor
  • 371 Views
  • 0 replies
  • 1 kudos

docs.azure.cn

https://docs.azure.cn/en-us/databricks/_static/notebooks/mlflow/mlflow-end-to-end-example-azure.htmlI imported the above notebook and try in Databricks community, but those subplots for Box plots are giving me errors as below:AttributeError ...

Machine Learning
Reply
  • 371 Views
  • 0 replies
  • 1 kudos
j_b
by • New Contributor
  • 958 Views
  • 2 replies
  • 0 kudos

API limit on mlflow.tracking.client.MlflowClient.list_run_infos method?

I'm trying out managed MLflow on Databricks Community edition, with tracking data saved on Databricks and artifacts saved on my own AWS S3 bucket. I created one experiment and logged 768 runs in the experiment. When I try to get the list of the runs ...

Machine Learning
Reply
  • 958 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @jae baak​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 0 kudos
1 More Replies
Labels