cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Sujitha
by • Community Manager
  • 3632 Views
  • 3 replies
  • 2 kudos

Resolved! Hello Community Users,  We recently announced a new Large Language Models (LLM) program, the first of its kind on edX! Learn how to develop production...

Hello Community Users, We recently announced a new Large Language Models (LLM) program, the first of its kind on edX! Learn how to develop production-ready LLM applications and dive into the theory behind foundation models. Taught by industry experts...

Machine Learning
Reply
  • 3632 Views
  • 3 replies
  • 2 kudos
View Replies
Latest Reply
APadmanabhan
Moderator
  • 2 kudos

Hi @163050 You could download the Dbc file from the course, we already have the LLM course in the Customer Academy.

  • 2 kudos
2 More Replies
HT
by • New Contributor II
  • 9713 Views
  • 5 replies
  • 2 kudos

Trying out Dolly - how to load pytorch_model.bin so it's not downloading it every time the cluster is restarted

Hi, I am new to LLM and am curious to try it out. I did the following code to test from the databricks website:import torch from transformers import pipeline instruct_pipeline = pipeline(model="databricks/dolly-v2-12b", torch_dtype=torch.bfloat16, tr...

Machine Learning
Reply
  • 9713 Views
  • 5 replies
  • 2 kudos
View Replies
Latest Reply
sean_owen
Honored Contributor II
  • 2 kudos

Just set the HF cache dir to a persistent path on /dbfs:import os os.environ['TRANSFORMERS_CACHE'] = "/dbfs/..."

  • 2 kudos
4 More Replies
JAHNAVI
by • New Contributor III
  • 6669 Views
  • 1 replies
  • 2 kudos

Resolved! error in running a LLM model in pyfunc.spark_udf

error in running a LLM model in pyfunc.spark_udf

Machine Learning
Reply
  • 6669 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
JAHNAVI
New Contributor III
  • 2 kudos

Solution:   Please find below the example. Creating a secret and scope is a one time activity once we create a scope and secret we can access the token using any notebook or cluster in the workspace as shown below. After creating a secret if we want ...

  • 2 kudos
Labels