Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
02-02-2025 04:00 AM
Hi @Nishat ,
It looks like there's a problem with GPU compability. As mentioned in the error message, FlashAttention only supports Ampere GPUs or newer.
According to following thread, GPU architecture you've chosen is not supported
RuntimeError: FlashAttention only supports Ampere GPUs or newer. · Issue #28188 · huggingface/transf...
You can try different compute, according to above github thread GPU Type: NVIDIA H100 NVL GPU should work