I'm trying to run the Dolly demo on a nc6s_v3 128 GB single node cluster but get the following out of memory. Does anyone have any suggestions as to why this is/how to solve it? Thx
OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 15.78 GiB total capacity; 14.48 GiB already allocated; 5.50 MiB free; 14.64 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
