cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

HT
by New Contributor II
  • 4636 Views
  • 5 replies
  • 2 kudos

Trying out Dolly - how to load pytorch_model.bin so it's not downloading it every time the cluster is restarted

Hi, I am new to LLM and am curious to try it out. I did the following code to test from the databricks website:import torch from transformers import pipeline instruct_pipeline = pipeline(model="databricks/dolly-v2-12b", torch_dtype=torch.bfloat16, tr...

  • 4636 Views
  • 5 replies
  • 2 kudos
Latest Reply
sean_owen
Honored Contributor II
  • 2 kudos

Just set the HF cache dir to a persistent path on /dbfs:import os os.environ['TRANSFORMERS_CACHE'] = "/dbfs/..."

  • 2 kudos
4 More Replies
findinpath
by Contributor
  • 3247 Views
  • 9 replies
  • 5 kudos

How to mount s3 bucket in community edition cluster?

I'm using Databricks Community Edition for testing purposes on a OSS project.I'm spinning up the cluster automatically through Databricks Clusters API.The automated tests rely on AWS S3 infrastructure, reason why I need to mount the S3 bucket on the ...

  • 3247 Views
  • 9 replies
  • 5 kudos
Latest Reply
findinpath
Contributor
  • 5 kudos

I haven't found any solution.I'm assuming that currently my only option is the usage of Databricks Enterprise to model scenarios involving the mounting of object storage buckets.

  • 5 kudos
8 More Replies
Labels