cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
sanjay
by • Valued Contributor II
  • 34761 Views
  • 2 replies
  • 1 kudos

Resolved! torch.cuda.OutOfMemoryError: CUDA out of memory

Hi,I am using pynote/whisper large model and trying to process data using spark UDF and getting following error.torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 172.00 MiB (GPU 0; 14.76 GiB total capacity; 6.07 GiB already allocated...

Machine Learning
Reply
  • 34761 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
JMTech18
New Contributor II
  • 1 kudos

Try to run these codesimport torchtorch.cuda.empty_cache()And make sure to find the optimize batch size otherwise the error can occur again

  • 1 kudos
1 More Replies
rusty
by • New Contributor II
  • 5159 Views
  • 2 replies
  • 2 kudos

Resolved! "Photon ran out of memory" while when trying to get the unique Id from sql query

I am trying to get all unique id from sql query and I always run out of memoryselect concat_ws(';',view.MATNR,view.WERKS) from hive_metastore.dqaas.temp_view as view join hive_metastore.dqaas.t_dqaas_marc as marc on marc.MATNR = view.MATNR where view...

Machine Learning
Reply
  • 5159 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Anil Kumar Chauhan​ We haven't heard from you since the last response from @Werner Stinckens​  . Kindly share the information with us, and in return, we will provide you with the necessary solution.Thanks and Regards

  • 2 kudos
1 More Replies
DataBRObin
by • New Contributor III
  • 4303 Views
  • 6 replies
  • 1 kudos

FFmpeg frame extraction explodes memory, how to mitigate?

For a computer vision project, my raw data consists of encrypted videos (60fps) stored in Azure Blob Storage. In order to have the data usable for model training, I need to do some preprocessing and for that I need the video split into individual fra...

Machine Learning
Reply
  • 4303 Views
  • 6 replies
  • 1 kudos
View Replies
Latest Reply
DataBRObin
New Contributor III
  • 1 kudos

In the end, I decided to change around the workflow so it is as efficient as I could imagine it:Extract frames of video files in a containerized application somewhere running ffmpeg and storing the resulting frames in a parquet file in blob storage (...

  • 1 kudos
5 More Replies
Labels