cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Ritchie
by New Contributor
  • 46 Views
  • 1 replies
  • 0 kudos

Unable to Use VectorAssembler in PySpark 3.5.0 Due to Whitelisting

Hi,I am currently using PySpark version 3.5.0 on my Databricks cluster. Despite setting the required configuration using the command: spark.conf.set("spark.databricks.ml.whitelist", "true"), I am still encountering an issue while trying to use the Ve...

  • 46 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @Ritchie, Can you run and validate outputs True: print(spark.conf.get("spark.databricks.ml.whitelist"))

  • 0 kudos
ash1127
by New Contributor
  • 3879 Views
  • 3 replies
  • 0 kudos

Permission Issue

 I want learn about Machine Learning operations but I can't access this page.https://www.databricks.com/training/catalog/advanced-machine-learning-operations-3508 Access deniedYou do not have permission to access this page, please contact your admini...

  • 3879 Views
  • 3 replies
  • 0 kudos
Latest Reply
Advika
Databricks Employee
  • 0 kudos

Hello, @ash1127!Please file a ticket with the Databricks support team to get assistance with this issue.Thank you.

  • 0 kudos
2 More Replies
ddpotapov
by New Contributor
  • 55 Views
  • 2 replies
  • 0 kudos

Model from code approach

Hi Databricks Team,I am trying to understand the "model from code" approach. I am reading your Big Book of MLOps.Is it correct that when using this approach I need to train the model twice - in development and in production?I am asking because in thi...

  • 55 Views
  • 2 replies
  • 0 kudos
Latest Reply
ddpotapov
New Contributor
  • 0 kudos

Thank you for your answer. You said:initially in the development environment as part of model developmentWhat does this mean?Usually, I take a model, run a lot of training experiments with different hyperparameters. And when I find the best parameter...

  • 0 kudos
1 More Replies
infinitylearnin
by New Contributor III
  • 44 Views
  • 0 replies
  • 0 kudos

Data practitioner in AI Era

As the AI revolution takes off in 2025, there is a renewed emphasis on adopting a Data-First approach. Organizations are increasingly recognizing the need to establish a robust data foundation while preparing a skilled fleet of Data Engineers to tack...

  • 44 Views
  • 0 replies
  • 0 kudos
Djay101
by New Contributor
  • 447 Views
  • 2 replies
  • 0 kudos

How do we log without a dbfs for MLFlow models.

Hi Databricks Team,We are planning a UC Migration for a customer who currently has around 500 experiments, each with multiple runs. These experiments are registered and MLflow is logging to DBFS locations. However, we have not found any documentation...

  • 447 Views
  • 2 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @rahuja / @Djay101, Do you have any account team contact for your Databricks subscription? I think this kind of migration requires more analysis based on your use-case. I could not find a straight-forward way to perform this migration, perhaps the...

  • 0 kudos
1 More Replies
kishan_
by New Contributor II
  • 781 Views
  • 3 replies
  • 2 kudos

Resolved! Facing issues with passing memory checkpointer in lanngraph agents

Hi,I am trying to create a simple langgraph agent in Databricks, the agent also uses lanngraph memory checkpoint which enables to store the state of the graph. This is working fine when I am trying it in Databricks notebook, but when I tried to log t...

Machine Learning
langgraph
mlflow
  • 781 Views
  • 3 replies
  • 2 kudos
Latest Reply
kishan_
New Contributor II
  • 2 kudos

@morenoj11 The solution which you have mentioned, Have you tried to deploy the same in Databricks model serving ?

  • 2 kudos
2 More Replies
magrat08
by New Contributor II
  • 17250 Views
  • 3 replies
  • 0 kudos

Resolved! Upload a file

Hi - I'm trying to upload a file, so that I can use the same in my notebook to try ML experiments with Databricks. From my workspace, I created a folder. But the option 'Create -> File' does not do anything. So not able to add any file. From a notebo...

  • 17250 Views
  • 3 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Can you try the approach mentioned in https://ganeshchandrasekaran.com/databricks-how-to-load-data-from-google-drive-github-c98d6b34d1b5 

  • 0 kudos
2 More Replies
JamesBrown54
by New Contributor
  • 829 Views
  • 2 replies
  • 0 kudos

Retrieve data from multiple .mdb files using Python.

Hello,I'm interested in accessing several .mdb Access files stored in either Azure Data Lake Storage (ADLS) or the Databricks File System using Python. Could you provide guidance on how to accomplish this? It would be immensely helpful if you could a...

  • 829 Views
  • 2 replies
  • 0 kudos
Latest Reply
NandiniN
Databricks Employee
  • 0 kudos

These are a couple of blogs and docs too https://docs.databricks.com/en/connect/storage/azure-storage.html

  • 0 kudos
1 More Replies
dkxxx-rc
by New Contributor III
  • 344 Views
  • 5 replies
  • 1 kudos

Resolved! Nested runs don't group correctly in MLflow

How do I get MLflow child runs to appear as children of their parent run in the MLflow GUI, if I'm choosing my own experiment location instead of letting everything be written to the default experiment location?If I run the standard tutorial (https:/...

dkxxxrc_0-1736289524445.png
  • 344 Views
  • 5 replies
  • 1 kudos
Latest Reply
dkxxx-rc
New Contributor III
  • 1 kudos

OK, here's more info about what's wrong, and a solution.I used additional parameter logging to determine that no matter how I adjust the parameters of the inner call to ```mlflow.start_run()```the `experiment_id` parameter of the child runs differs f...

  • 1 kudos
4 More Replies
llmnerd
by New Contributor
  • 318 Views
  • 1 replies
  • 0 kudos

UDF LLM DataBrick pickle error

Hi there,I am trying to parellize a text extraction via the Databrick foundational model.Any pointers to suggestions or examples are welcomeThe code and error below.model = "databricks-meta-llama-3-1-70b-instruct" temperature=0.0 max_tokens=1024 sch...

  • 318 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ayushi_Suthar
Databricks Employee
  • 0 kudos

Hi @llmnerd , Hope you are doing well!  Upon reviewing the details provided, we have identified several observations regarding the SparkContext serialization error encountered. Please find a detailed analysis and our recommendations below:==== ANALYS...

  • 0 kudos
cwij95
by New Contributor III
  • 872 Views
  • 8 replies
  • 4 kudos

`mlflow.login()` failed with error: Failed to validate databricks credentials: Basic Authentication

I have been using mlflow with Databricks community edition for 3 months without any issue. However, today when I tried to login to the mlflow host (https://community.cloud.databricks.com/), using this code, I keep getting the error below. # Setup MLf...

  • 872 Views
  • 8 replies
  • 4 kudos
Latest Reply
cwij95
New Contributor III
  • 4 kudos

Thank you for the update. It seems the provided reference https://docs.databricks.com/en/dev-tools/auth/oauth-u2m.html discusses authenticating access to the Databricks Platform (not the Databricks CE), where we need an Account ID to proceed with the...

  • 4 kudos
7 More Replies
dkxxx-rc
by New Contributor III
  • 442 Views
  • 2 replies
  • 1 kudos

Resolved! Save model from AutoML to MLflow in LightGBM flavor

I want to get the LightGBM built-in variable importance values from a model that was generated by AutoML.  That's not logged in the metrics by default - can I change a setting so that it will be logged?More fundamentally:  what I'd really like is to ...

  • 442 Views
  • 2 replies
  • 1 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 1 kudos

Additional Considerations The pyfunc.add_to_model() function you mentioned is used to add the Python Function flavor to the model, which is different from changing the primary flavor of the logged model. That's why changing its parameter didn't solve...

  • 1 kudos
1 More Replies
AChang
by New Contributor III
  • 4302 Views
  • 1 replies
  • 1 kudos

MlflowException: Unable to download model artifacts in Databricks while registering model with MLflo

I am attempting to log, register, and deploy a finetuned GPT2 model in Databricks. While I have been able to get my logging code to run, when I try to run my registration code, I get an MlflowException error.Here is my model logging code.mlflow.set_r...

  • 4302 Views
  • 1 replies
  • 1 kudos
Latest Reply
TimoLeco_56656
New Contributor II
  • 1 kudos

I've experience the same error. The issue is that the model uri is not correct.The model is logged with:mlflow.transformers.log_model( ... , artifact_path="gpt2", ...)The artifact_path is the last part of the model uri. If you don't specify it, it's ...

  • 1 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels