- 2164 Views
- 2 replies
- 0 kudos
Upgrading cuDNN on Databricks notebook
I'm trying to upgrade Tensorflow version from 2.8 to 2.13 on Databricks notebook that is attached to a cluster with Databricks Runtime 10.4. How can I upgrade cuDNN from 8.0 to at least 8.6 to be compatible with the Tensorflow new version?
- 2164 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Retired_mod , Thanks for your response. When I run '!conda list cudnn' on databricks notebook, I get the following error: '/bin/bash: conda: command not found'
- 0 kudos
- 2406 Views
- 0 replies
- 0 kudos
OutOfMemoryError: CUDA out of memory on LLM Finetuning
I am trying to finetune llama2_lora model using the xTuring library, while facing this error. (batch size is 1). I am working on a cluster having 1 Worker (28 GB Memory, 4 Cores) and 1 Driver (110 GB Memory, 16 Cores). I am facing this error: OutOfMe...
- 2406 Views
- 0 replies
- 0 kudos
- 1828 Views
- 0 replies
- 0 kudos
Error in Tensorflow training job
I upgraded Tensorflow on Databricks notebook using %pip command. Now when running the training job, I get this error: "DNN library initialization failed."
- 1828 Views
- 0 replies
- 0 kudos
- 1706 Views
- 1 replies
- 0 kudos
Using AutoML to predict completion dates of a project management dataset
Hello! I am fairly new to Databricks. I'm trying to do a proof of concept with AutoML in Databricks at my organization, and the dataset I am using is a project management dataset. Here's a sample: project_idmarketgeneral_contractorproject_typepermit_...
- 1706 Views
- 1 replies
- 0 kudos
- 7360 Views
- 8 replies
- 2 kudos
Resolved! Scalable ML course error on Lab Setup (Community Edition)
Hello,I am trying to complete the exercises of the course "Scalable Machine Learning with Apache Spark" using Databricks Community Edition, but when I run the Lab Setup I get the following error:HTTPError: 503 Server Error: Service Unavailable for ur...
- 7360 Views
- 8 replies
- 2 kudos
- 2 kudos
I'm experiencing the same issue while using community edition for this classroom: https://github.com/databricks-academy/large-language-models. What subscription level do I upgrade to?
- 2 kudos
- 4326 Views
- 3 replies
- 0 kudos
Resolved! An error occurred while loading the model. Failed to load the pickled function from a hexadecimal
[8586fsbgpb] An error occurred while loading the model. Failed to load the pickled function from a hexadecimal string. Error: Can't get attribute 'transform_input' on <module '__main__' from '/opt/conda/envs/mlflow-env/bin/gunicorn'>.I´m using the fu...
- 4326 Views
- 3 replies
- 0 kudos
- 0 kudos
However I could not progress in the end I mean because I found the error I reported in other thread as follows[5bb99fzs2f] An error occurred while loading the model. You haven't configured the CLI yet! Please configure by entering `/opt/conda/envs/ml...
- 0 kudos
- 4746 Views
- 7 replies
- 0 kudos
Resolved! Why java is no included in notebooks
Hi All,is there any way to use java in notebook ?
- 4746 Views
- 7 replies
- 0 kudos
- 0 kudos
yes spark-submit in jar will work,it is available in the jobs/workflow section in the databricks
- 0 kudos
- 1648 Views
- 1 replies
- 1 kudos
Resolved! Using AutoML in Azure Databricks with a shared cluster
Do you have to use AutoML in Azure Databricks on a personal compute cluster or can you use a shared cluster?Can you point me to some documentation that supports the statement that you can run AutoML on a shared cluster with Azure Databricks?
- 1648 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @david_stroud Greetings! AutoML is not supported on Shared clusters. Please check the documentation below https://learn.microsoft.com/en-us/azure/databricks/machine-learning/automl/#--requirements
- 1 kudos
- 11335 Views
- 4 replies
- 1 kudos
Mlflowexception: "Connection broken: ConnectionResetError(104, \\\'Connection reset by peer\\\')"
Hello,I have a workflow running which from time to time crashes with the error:MlflowException: The following failures occurred while downloading one or more artifacts from models:/incubator-forecast-charging-demand-power-and-io-dk2/Production: {'pyt...
- 11335 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @cl2,Thanks for bringing up your concerns; always happy to help Upon going through the details, it appears there was an HTTP connection error downloading artifacts. This typically shouldn’t happen, but it can occur intermittently as a transient n...
- 1 kudos
- 6264 Views
- 6 replies
- 3 kudos
`collect()`ing Large Datasets in R
Background: I'm working on a pilot project to assess the pros and cons of using DataBricks to train models using R. I am using a dataset that occupies about 5.7GB of memory when loaded into a pandas dataframe. The data are stored in a delta table in ...
- 6264 Views
- 6 replies
- 3 kudos
- 3 kudos
@acsmaggart Please try using collect_larger() to collect the larger dataset. This should work. Please refer to the following document for more info on the library.https://medium.com/@NotZacDavies/collecting-large-results-with-sparklyr-8256a0370ec6
- 3 kudos
- 2553 Views
- 2 replies
- 5 kudos
How can I view the storage space taken by a registered model using MLFlow?
The information viewed about the registered models on the Models tab is very minimal. Just showing the tags we pass in and version information. How can I get more details about the model such as the size on disk?
- 2553 Views
- 2 replies
- 5 kudos
- 5 kudos
Hi,I have used the MLFlow client, but I am not sure where to find the size of the model image.The response to client.search_registered_models() I am getting is the following:<RegisteredModel: aliases={}, creation_timestamp=17061..., description='', l...
- 5 kudos
- 11629 Views
- 5 replies
- 0 kudos
Resolved! Model serving endpoint requires workspace-access entitlement?
Hi all, is anyone getting status 403 when requesting a model serving endpoint with error message "This API is disabled for users without the workspace-access entitlement"? I am accessing my model serving endpoint with a service principal access token...
- 11629 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @run480 , We understand that you are facing the following error while you are trying to access the model serving endpoint with a Service Principal Access Token: ++++++++++++++++++++++++++++++++++++++ "This API is disabled for users without the wor...
- 0 kudos
- 1722 Views
- 0 replies
- 0 kudos
Editing posts
Hey,I have a post that I would like to edit. However, when I use the drop down menu there is no possibility to edit or delete the post - can someone help?
- 1722 Views
- 0 replies
- 0 kudos
- 1558 Views
- 0 replies
- 0 kudos
Loading Pre-trained Models in Databricks
Hello talented members of the community,I'm a very new Databricks user so please bear with me.I'm building a description matcher which uses a pre-trained model (universal-sentence-encoder). How can I load and use this model in my Databricks python no...
- 1558 Views
- 0 replies
- 0 kudos
- 7760 Views
- 0 replies
- 0 kudos
Ready to elevate your data insights to new heights? Discover the power of Databricks SQL Materialize
In our latest blog post, unlock the secrets to delivering fresh data and actionable insights straight to your business. Learn how Lakeview dashboards, now available on the Databricks Data Intelligence Platform, revolutionize visualization and reporti...
- 7760 Views
- 0 replies
- 0 kudos
