Download Dolly model on local machine

FutureLegend · ‎08-02-2023

Hi~ I am new to LLM engineering, and am trying to download the Dolly-v2-7b model on local machine, so I don't need to connect to internet each time I am going to run the Dolly-v2-7b. Is it possible to do that? Thanks a lot!

sean_owen · ‎08-04-2023

@Retired_mod this is not necessary for downloading a model; it actually has nothing to do with Databricks.

Dolly is hosted on Hugging Face. If you download it via transformers, it will actually cache it and not download again, already. But you can always just download the files of any model to a local dir: https://huggingface.co/databricks/dolly-v2-12b/tree/main You then load that local dir as a model.

FutureLegend · ‎08-07-2023

Hi Kaniz and Sean, thanks for your responses and time.
I was trying Kaniz's method, but got a reply from Sean, so I tried that too. I downloaded the file from the link Sean provided and saved it on my local machine, then used the code for Dollyv2 (https://huggingface.co/databricks/dolly-v2-3b) and ran it successfully without connecting to Huggingface/ Databricks. Haven't tried without internet, but I think it will work too. Thanks again Kaniz and Sean for your replies!

code I used:

pip install "accelerate>=0.16.0,<1" "transformers[torch]>=4.28.1,<5" "torch>=1.13.1,<2"

import torch

from transformers import pipeline

generate_text = pipeline(model="local_machine/path/to/downloaded_Dolly_folder", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")

res = generate_text("Explain to me the difference between nuclear fission and fusion.")

print(res[0]["generated_text"])

View solution in original post