Databricks Community

m_koch_unify · 07-20-2023

Hi all,I was following the hugging face model https://huggingface.co/TheBloke/Llama-2-70B-chat-GPTQ, which points to use Exllama (https://github.com/turboderp/exllama/), which has 4 bit quantization.Running on a A10-Single-GPU-64GB,I've cloned the Ex...

m_koch_unify · 07-21-2023

Hi @Kumaran,Thanks so much for the quick reply. When I run the script with !bash install_cusparse.shIt runs for a bit, but ultimately encounters an error. When I run !ls -l, i dont even see a data-mle directory in dbfshere is the full output from run...

Databricks Community

User Stats

User Activity

Running test inference on Llama-2-70B-chat-GPTQ… are C++ libraries installed correctly?

Re: Running test inference on Llama-2-70B-chat-GPTQ… are C++ libraries installed correctly?