szymon_dybczak
Esteemed Contributor III

Hi @Nishat ,

It looks like there's a problem with GPU compability. As mentioned in the error message, FlashAttention only supports Ampere GPUs or newer.
According to following thread, GPU architecture you've chosen is not supported
RuntimeError: FlashAttention only supports Ampere GPUs or newer. · Issue #28188 · huggingface/transf...

szymon_dybczak_0-1738497458699.png

You can try different compute, according to above github thread GPU Type: NVIDIA H100 NVL GPU should work