Hi @Nishat ,
It looks like there's a problem with GPU compability. As mentioned in the error message, FlashAttention only supports Ampere GPUs or newer.
According to following thread, GPU architecture you've chosen is not supported
RuntimeError: FlashAttention only supports Ampere GPUs or newer. · Issue #28188 · huggingface/transf...
You can try different compute, according to above github thread GPU Type: NVIDIA H100 NVL GPU should work
