cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

dkxxx-rc
by New Contributor III
  • 518 Views
  • 5 replies
  • 1 kudos

Resolved! Nested runs don't group correctly in MLflow

How do I get MLflow child runs to appear as children of their parent run in the MLflow GUI, if I'm choosing my own experiment location instead of letting everything be written to the default experiment location?If I run the standard tutorial (https:/...

dkxxxrc_0-1736289524445.png
  • 518 Views
  • 5 replies
  • 1 kudos
Latest Reply
dkxxx-rc
New Contributor III
  • 1 kudos

OK, here's more info about what's wrong, and a solution.I used additional parameter logging to determine that no matter how I adjust the parameters of the inner call to ```mlflow.start_run()```the `experiment_id` parameter of the child runs differs f...

  • 1 kudos
4 More Replies
llmnerd
by New Contributor
  • 389 Views
  • 1 replies
  • 0 kudos

UDF LLM DataBrick pickle error

Hi there,I am trying to parellize a text extraction via the Databrick foundational model.Any pointers to suggestions or examples are welcomeThe code and error below.model = "databricks-meta-llama-3-1-70b-instruct" temperature=0.0 max_tokens=1024 sch...

  • 389 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ayushi_Suthar
Databricks Employee
  • 0 kudos

Hi @llmnerd , Hope you are doing well!  Upon reviewing the details provided, we have identified several observations regarding the SparkContext serialization error encountered. Please find a detailed analysis and our recommendations below:==== ANALYS...

  • 0 kudos
cwij95
by New Contributor III
  • 1047 Views
  • 8 replies
  • 4 kudos

`mlflow.login()` failed with error: Failed to validate databricks credentials: Basic Authentication

I have been using mlflow with Databricks community edition for 3 months without any issue. However, today when I tried to login to the mlflow host (https://community.cloud.databricks.com/), using this code, I keep getting the error below. # Setup MLf...

  • 1047 Views
  • 8 replies
  • 4 kudos
Latest Reply
cwij95
New Contributor III
  • 4 kudos

Thank you for the update. It seems the provided reference https://docs.databricks.com/en/dev-tools/auth/oauth-u2m.html discusses authenticating access to the Databricks Platform (not the Databricks CE), where we need an Account ID to proceed with the...

  • 4 kudos
7 More Replies
dkxxx-rc
by New Contributor III
  • 552 Views
  • 2 replies
  • 1 kudos

Resolved! Save model from AutoML to MLflow in LightGBM flavor

I want to get the LightGBM built-in variable importance values from a model that was generated by AutoML.  That's not logged in the metrics by default - can I change a setting so that it will be logged?More fundamentally:  what I'd really like is to ...

  • 552 Views
  • 2 replies
  • 1 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 1 kudos

Additional Considerations The pyfunc.add_to_model() function you mentioned is used to add the Python Function flavor to the model, which is different from changing the primary flavor of the logged model. That's why changing its parameter didn't solve...

  • 1 kudos
1 More Replies
AChang
by New Contributor III
  • 4469 Views
  • 1 replies
  • 1 kudos

MlflowException: Unable to download model artifacts in Databricks while registering model with MLflo

I am attempting to log, register, and deploy a finetuned GPT2 model in Databricks. While I have been able to get my logging code to run, when I try to run my registration code, I get an MlflowException error.Here is my model logging code.mlflow.set_r...

  • 4469 Views
  • 1 replies
  • 1 kudos
Latest Reply
TimoLeco_56656
New Contributor II
  • 1 kudos

I've experience the same error. The issue is that the model uri is not correct.The model is logged with:mlflow.transformers.log_model( ... , artifact_path="gpt2", ...)The artifact_path is the last part of the model uri. If you don't specify it, it's ...

  • 1 kudos
Akash_Wadhankar
by New Contributor III
  • 478 Views
  • 0 replies
  • 0 kudos

Learn Databricks AI medium article series for fellow learners.

When it comes to machine learning, the platform plays a pivotal role in successful implementation. Databricks offers a best-in-class machine learning platform with cutting-edge features such as MLflow, Model Registry, Feature Store, and MLOps, which ...

Machine Learning
DatabricksML MachineLearning AI FeatureStore DecisionScience
  • 478 Views
  • 0 replies
  • 0 kudos
nicobuko
by New Contributor III
  • 1389 Views
  • 13 replies
  • 2 kudos

MLflow: Connect python with Community Edition

Hello,I am new to databricks and want to work with MLFlow in the Databricks Community Edition. In python i am using mlflow.login(). This requests me to enter a password. But i do not have any password due to the fact that databricks login only requir...

  • 1389 Views
  • 13 replies
  • 2 kudos
Latest Reply
Walter_C
Databricks Employee
  • 2 kudos

I am currently looking with our internal teams if this will be provided in the near future, still waiting for confirmation.

  • 2 kudos
12 More Replies
sjohnston2
by New Contributor II
  • 666 Views
  • 2 replies
  • 2 kudos

Resolved! XGBoost Feature Weighting

We are trying to train a predictive ML model using the XGBoost Classifier. Part of the requirements we have gotten from our business team is to implement feature weighting as they have defined certain features mattering more than others. We have 69 f...

  • 666 Views
  • 2 replies
  • 2 kudos
Latest Reply
Walter_C
Databricks Employee
  • 2 kudos

Hello @sjohnston2 here is some information i found internally: Possible Causes Memory Access Issue: The segmentation fault suggests that the program is trying to access memory that it's not allowed to, which could be caused by an internal bug in XGBo...

  • 2 kudos
1 More Replies
AnkurMittal008
by New Contributor III
  • 577 Views
  • 2 replies
  • 1 kudos

Resolved! Online Feature Table : Storage

Databricks Online Feature Table is in public preview , And we have some questions on this 1) What storage is being used for Online Feature table's Data. Our offline feature table is stored in Unity Catalog managed S3 bucket (Customer AWS ). Does onli...

  • 577 Views
  • 2 replies
  • 1 kudos
Latest Reply
AnkurMittal008
New Contributor III
  • 1 kudos

Thanks a lot @Walter_C ..

  • 1 kudos
1 More Replies
mharrison
by New Contributor II
  • 467 Views
  • 1 replies
  • 1 kudos

Resolved! No Spark Session Available Within Model Serving Environment

Hi,Is it possible to have a Spark session, that can be to used query the Unity Catalog etc, available within a Model Serving?I have an MLFlow Pyfunc model that needs to get data from a Feature Table as part of its `.predict()` method. See my earlier ...

  • 467 Views
  • 1 replies
  • 1 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 1 kudos

Hi @mharrison  Creating a Spark session within a Model Serving environment is not directly supported, which is why you are encountering the Exception: No SparkSession Available! error. This limitation arises because the serving environment does not a...

  • 1 kudos
byrnesy5
by New Contributor II
  • 26419 Views
  • 3 replies
  • 0 kudos

dbfs not found

Hi, I've saved a custom pyfunc and now I'm trying to load it in a pandas_udf. It works on small samples or if I repartition everything to 1 partition, but when I try to run it on a larger sample and distribute it across my cluster it fails repeatably...

  • 26419 Views
  • 3 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

This problem can often be attributed to the model artifacts not being available on all the executors, especially in a distributed environment. Can you try using the dbutils.fs.refreshMounts() in your code? If the model is small enough, broadcast it t...

  • 0 kudos
2 More Replies
mharrison
by New Contributor II
  • 503 Views
  • 2 replies
  • 0 kudos

Feature Lookup Help

Hi,ContextI'm looking for help trying to get Unity Catalog Feature Lookup to work with my model how I need it to.I have a trained darts time series model that takes as input to its `.predict()` method both the history of the variable in question, and...

  • 503 Views
  • 2 replies
  • 0 kudos
Latest Reply
mharrison
New Contributor II
  • 0 kudos

Thanks for your response. It sounds like the 2nd approach is best for me, modifying the `predict()` method to perform the required history lookup.Is it possible to do this via the Feature Engineering client within that method, or should I simply quer...

  • 0 kudos
1 More Replies
johndoe99012
by New Contributor II
  • 600 Views
  • 4 replies
  • 1 kudos

How to serve a Unity Catalog ML model to external usage

Hello everyone I am following this notebook tutorial https://docs.databricks.com/en/machine-learning/manage-model-lifecycle/index.html#example-notebook Now I can register a machine learning model in Unity Catalog, but the tutorial only shows how to u...

  • 600 Views
  • 4 replies
  • 1 kudos
Latest Reply
filipniziol
Contributor III
  • 1 kudos

Hi @johndoe99012 If the answer resolved your question, please consider marking it as the solution. It helps others in the community find answers more easily.  

  • 1 kudos
3 More Replies
AntonDBUser
by New Contributor III
  • 1220 Views
  • 5 replies
  • 1 kudos

Hosting R apps and models

Hello!Has anyone tried to host an R Shiny app using Databricks apps? It is very useful for Python apps, but it would be nice to know if anyone have any workaround to make it work with R Shiny Apps.We also wonder about Databricks model hosting for R m...

  • 1220 Views
  • 5 replies
  • 1 kudos
Latest Reply
parthSundarka
Databricks Employee
  • 1 kudos

Hi @AntonDBUser, Did you get a chance to test this? Let me know if the above solution can be marked as accepted. Thanks

  • 1 kudos
4 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels