cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Science & Machine Learning

Forum Posts

MohsenJ
by Contributor
  • 856 Views
  • 2 replies
  • 1 kudos

Model Lineage with Feature Engineering is missing tables and notebooks

I am trying to track the lineage of model and tables using the FeatureEngineeringClient. The table lineage shows the relevant tables and notebooks but the model lineage shows only the model. No notebook and tables. here is my code  fe = FeatureEngine...

  • 856 Views
  • 2 replies
  • 1 kudos
Latest Reply
MohsenJ
Contributor
  • 1 kudos

ok I realized something else. That although I used FeatureEngineeringCient, MLflow model artifact suggest I used FeatureStoreClient. Please see attachment.   

  • 1 kudos
1 More Replies
Rob_S
by New Contributor III
  • 6302 Views
  • 7 replies
  • 6 kudos

Displaying graphviz images in a notebook

Hi,I'm experimenting with process mining in a Databricks notebook using the OSS library PM4PY. I've been working through some tutorials and the notebook they provide on Github:https://github.com/pm4py/pm4py-core/blob/release/notebooks/3_process_disco...

  • 6302 Views
  • 7 replies
  • 6 kudos
Latest Reply
rushank29
New Contributor II
  • 6 kudos

@Rob_S i am also in the same situation the code cell executes but no visualization how did you tackle this problem?

  • 6 kudos
6 More Replies
julia
by New Contributor II
  • 1797 Views
  • 3 replies
  • 0 kudos

parallel execution with applyinpandas on partitioned table

Hi,I have a job that uses df.groupby(“Country”).applyInPandas(..) to run pandas-based hyperparameter tuning in parallel for 6 countries.It runs on a cluster with 4 workers (Chosen like this because the countries’ datasets are of different sizes – so ...

  • 1797 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @julia, Here are a few reference links:-  https://spark.apache.org/docs/3.1.1/api/python/reference/api/pyspark.sql.PandasCogroupedOps.applyInPandas.htmlhttps://docs.databricks.com/en/pandas/pandas-function-apis.htmlhttps://api-docs.databricks.com/...

  • 0 kudos
2 More Replies
ombhuyan
by New Contributor II
  • 4118 Views
  • 6 replies
  • 3 kudos

Serving API endpoint failing

Hi Team,I registered my ML model in databricks but while trying to serve an API endpoint for the model it is failing with the following error logs.Service logs: There are currently no replicas in a running state.Build logs :Build never started - chec...

  • 4118 Views
  • 6 replies
  • 3 kudos
Latest Reply
Annapurna_Hiriy
New Contributor III
  • 3 kudos

@ombhuyan We currently only upload logs during the build phase to the user (i.e where we install the pip dependencies) but we don't upload logs during the pre-build phase (i.e where we download the model). That's why you may not see clear error messa...

  • 3 kudos
5 More Replies
SR_71
by New Contributor II
  • 5459 Views
  • 6 replies
  • 3 kudos

Databricks Notebook Rendering Issue: IPython.lib.display.IFrame

Similar issue here: https://stackoverflow.com/questions/71336374/randomforestclassifier-explainer-dashboard-output-in-databricks-notebook-is-notActual output – Databricks Notebook Expected Output – Jupyter Notebook Reproducible Code Example#pip insta...

image image
  • 5459 Views
  • 6 replies
  • 3 kudos
Latest Reply
ChanduBhujang
New Contributor II
  • 3 kudos

Hi Abhishek, I followed your steps, I am having in identifying the dashboard link. How do I figure out the first two words dbc-dp- for my cluster? 

  • 3 kudos
5 More Replies
TomBurns
by New Contributor
  • 760 Views
  • 1 replies
  • 0 kudos

Identity Resolution

Looking for best solutions for identity resolution. I already have deterministic matching. Exploring probabilistic solutions. Any advice for me?

  • 760 Views
  • 1 replies
  • 0 kudos
Latest Reply
MFGorin
New Contributor II
  • 0 kudos

Recommend checking out Amperity. Listed on Databricks marketplace, support delta sharing and unity catalog. Patented AI approach to ID resolution https://docs.amperity.com/stitch.html

  • 0 kudos
HHYOOOO
by New Contributor III
  • 1495 Views
  • 1 replies
  • 0 kudos

Resolved! Github Datasets/Labs for Large Language Models: Application through Production is not working

I've signed up for the module for certification on Large Language Models: Application through Production.Follow the Github instructions and install the notebooks provided.Unfortunately none of the workbooks are working due to the- Badly setup file pa...

correct.JPG Errort.JPG
  • 1495 Views
  • 1 replies
  • 0 kudos
Latest Reply
HHYOOOO
New Contributor III
  • 0 kudos

No further instructions on the Read-me here: https://github.com/databricks-academy/large-language-models/tree/publishedFollowed all the setup steps, but the file paths in /include are not working fine.Why does not Databricks provide the direct links ...

  • 0 kudos
Badarla
by New Contributor
  • 2775 Views
  • 3 replies
  • 2 kudos

Customize mail notification from Databricks workflow

Hi All,Can we customize the mail subject and body that we receive from Azure Databricks workflow upon failure jobs? Kindly help me, if we can do so.Thanks,Moshe

  • 2775 Views
  • 3 replies
  • 2 kudos
Latest Reply
np75
New Contributor II
  • 2 kudos

I have three workspaces and the alerts sent by the jobs running are not referencing the workspace for example. So if I run the job to dev environemnt I get an alert like if the job has been executed from the prod. This si a huge issue for our admins....

  • 2 kudos
2 More Replies
sqlshep
by New Contributor III
  • 2393 Views
  • 3 replies
  • 1 kudos

InvalidConfigurationError: You haven't configured the CLI yet! Please configure by entering ...

Running a python function in the notebook, i am getting the following InvalidConfigurationError: You haven't configured the CLI yet! Please configure by entering `/databricks/python_shell/scripts/db_ipykernel_launcher.py configure`When i try to run...

  • 2393 Views
  • 3 replies
  • 1 kudos
Latest Reply
wicked-lion
New Contributor II
  • 1 kudos

Facing the same issuefor me the error comes up when mlflow.get_experiment_by_name is called.I am  running a custom docker image built on databricksruntime/standard:13.3-LTScustom image so my packages are installed.

  • 1 kudos
2 More Replies
Octavian1
by Contributor
  • 6110 Views
  • 8 replies
  • 0 kudos

Resolved! Download model artifacts from MLflow

I am trying to find a way to locally download the model artifacts that build a chatbot chain registered with MLflow in Databricks, so that I can preserve the whole structure (chain -> model -> steps -> yaml & pkl files).There is a mention in a contri...

Octavian1_0-1708506098526.png
  • 6110 Views
  • 8 replies
  • 0 kudos
Latest Reply
Octavian1
Contributor
  • 0 kudos

  OK, eventually I found a solution. I write it below, whether somebody will need it. Basically, if in the download_artifacts method the local directory is an existing and accessible one in the DBFS, the process will work as expected.import os # Con...

  • 0 kudos
7 More Replies
amitca71
by Contributor II
  • 1164 Views
  • 2 replies
  • 0 kudos

mlflow.exceptions.MlflowException - Invalid metric 'refreshableTokenNotFound'

Hi,We are facing an mlflow.exceptions.MlflowException when mlflow is called from stream,when we load the model outside the stream, its loaded fine, while when we load it from within stream it fails with exception. to emphasize that it was working til...

  • 1164 Views
  • 2 replies
  • 0 kudos
Latest Reply
amitca71
Contributor II
  • 0 kudos

Downgrade to version 13.3 did the trick

  • 0 kudos
1 More Replies
GKH
by New Contributor II
  • 1774 Views
  • 1 replies
  • 1 kudos

Errors using Dolly Deployed as a REST API

We have deployed Dolly (https://huggingface.co/databricks/dolly-v2-3b) as a REST API endpoint on our infrastructure. The notebook we used to do this is included in the text below my question.The Databricks infra used had the following config -  (13.2...

  • 1774 Views
  • 1 replies
  • 1 kudos
Latest Reply
marcelo2108
Contributor
  • 1 kudos

I had a similar problem when I used HuggingFacePipeline(pipeline=generate_text) with langchain. It worked to me when I tried to use HuggingFaceHub instead. I used the same dolly-3b model.

  • 1 kudos
marcelo2108
by Contributor
  • 1526 Views
  • 2 replies
  • 0 kudos

Resolved! 0: 'error: TypeError("\'NoneType\' object is not callable") in api_request_parallel_processor.py

I´m facing this exception after use mlflow.langchain.log_model and test the logged model using the following commandprint(loaded_model.predict([{"query": "how does the performance of llama 2 compare to other local LLMs?"}]))tasks failed. Errors: {0: ...

  • 1526 Views
  • 2 replies
  • 0 kudos
Latest Reply
marcelo2108
Contributor
  • 0 kudos

I verified all steps @Kaniz_Fatma  and the objects and structure were looking good. As far as I understood on tests. Langchain Rag features such as RetrievalQA.from_chain_type does not work well with llm = HuggingFacePipeline instantiation steps. The...

  • 0 kudos
1 More Replies
sideshowBob1337
by New Contributor II
  • 842 Views
  • 3 replies
  • 0 kudos

Input training dataset field empty in Configure AutoML experiment

Trying to start an ML experiment on data in an extant metastore within a catalogue (SQL querys run fine on the database).  I can start an ML cluster, then attempt to start an AutoML expirement but I get stuck selecting training data - there are no da...

  • 842 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hey there! Thanks a bunch for being part of our awesome community!  We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solution...

  • 0 kudos
2 More Replies
hv129
by New Contributor
  • 1340 Views
  • 1 replies
  • 0 kudos

OutOfMemoryError: CUDA out of memory on LLM Finetuning

I am trying to finetune llama2_lora model using the xTuring library, while facing this error. (batch size is 1). I am working on a cluster having 1 Worker (28 GB Memory, 4 Cores) and 1 Driver (110 GB Memory, 16 Cores). I am facing this error: OutOfMe...

  • 1340 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @hv129, The error message you’re encountering indicates that your CUDA memory is running out while trying to allocate additional memory for your model. Let’s break down the details: Total Capacity: The 15.57 GiB mentioned in the error message ...

  • 0 kudos
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels