Hi,
I am using pynote/whisper large model and trying to process data using spark UDF and getting following error.
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 172.00 MiB (GPU 0; 14.76 GiB total capacity; 6.07 GiB already allocated; 120.75 MiB free; 6.25 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
Job is configured with 11.3 LTS ML with 1-8 instances of G4dn.4xlarge cluster.
Appreciate if you can provide any help.
Regards,
Sanjay