The configuration of my compute is
15.4 LTS ML (includes Apache Spark 3.5.0, GPU, Scala 2.12)
Standard_NC8as_T4_v3 on Azure Databricks
Hi @Nishat ,
It looks like there's a problem with GPU compability. As mentioned in the error message, FlashAttention only supports Ampere GPUs or newer.
According to following thread, GPU architecture you've chosen is not supported
RuntimeError: FlashAttention only supports Ampere GPUs or newer. · Issue #28188 · huggingface/transf...
You can try different compute, according to above github thread GPU Type: NVIDIA H100 NVL GPU should work
