cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Using Qwen with vLLM

pfzoz
Visitor

There are many conflict and dependency issues when trying to install VLLM and use the Qwen models (on serverless), even the v2 families.
I tried following this guide https://docs.databricks.com/aws/en/machine-learning/sgc-examples/tutorials/sgc-raydata-vllm
It lists very specific module versions to install. But VLLM normally fails to inspect the model architecture of Qwen. Is there an easier way to use these models (Specifically the VL-Instruct ones)?

0 REPLIES 0