cancel
Showing results for 
Search instead for 
Did you mean: 
Generative AI
Explore discussions on generative artificial intelligence techniques and applications within the Databricks Community. Share ideas, challenges, and breakthroughs in this cutting-edge field.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Sujitha
by • Databricks Employee
  • 42420 Views
  • 24 replies
  • 51 kudos

Databricks Announces the Industry’s First Generative AI Engineer Learning Pathway and Certification

Today, we are announcing the industry's first Generative AI Engineer learning pathway and certification to help ensure that data and AI practitioners have the resources to be successful with generative AI. At Databricks, we recognize that generative ...

Screenshot 2024-01-24 at 11.32.01 PM.png
  • 42420 Views
  • 24 replies
  • 51 kudos
Latest Reply
ThiamLee
Contributor III
  • 51 kudos

Great initiative by Databricks! This is a valuable opportunity to build practical GenAI skills and stay ahead in the rapidly evolving AI space.

  • 51 kudos
23 More Replies
maltasa
by • New Contributor II
  • 2015 Views
  • 1 replies
  • 0 kudos

Large datasets in Databricks

How can I efficiently handle large datasets in Databricks when performing group-by operations to avoid out-of-memory errors? Are there any best practices or optimizations for improving performance, such as partitioning or caching, especially when wor...

  • 2015 Views
  • 1 replies
  • 0 kudos
Latest Reply
Takuya-Omi
Valued Contributor III
  • 0 kudos

Hi, @maltasa I believe this article might help answer your question.Comprehensive Guide to Optimize Databricks, Spark and Delta Lake Workloads 

  • 0 kudos
Sujitha
by • Databricks Employee
  • 2502 Views
  • 1 replies
  • 2 kudos

Insights from a global survey of 1,100 technologists and interviews with 28 CIOs

How to unlock enterprise AI: “You can have all the AI in the world, but if it’s on a shaky data foundation, then it’s not going to bring you any value.” — Carol Clements, Chief Digital and Technology Officer, JetBlue Companies everywhere have been qu...

Screenshot 2024-11-15 at 3.52.14 AM.png
  • 2502 Views
  • 1 replies
  • 2 kudos
Latest Reply
Ajay-Pandey
Databricks MVP
  • 2 kudos

Must read

  • 2 kudos
llmnerd
by • New Contributor
  • 2373 Views
  • 1 replies
  • 1 kudos

DataBricks Foundational model rate limiting approach

Hi there,is this the correct approach to fulfil the rate limit restrictions in the foundational model API?from langchain_core.rate_limiters import InMemoryRateLimiter rate_limiter = InMemoryRateLimiter( requests_per_second=2.0, check_every_...

  • 2373 Views
  • 1 replies
  • 1 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 1 kudos

Hello @llmnerd, Yes, the approach you have outlined to fulfill the rate limit restrictions in the foundational model API using InMemoryRateLimiter from langchain_core appears to be correct. This setup should help you manage the rate limits effectivel...

  • 1 kudos
gwfj
by • New Contributor
  • 4471 Views
  • 2 replies
  • 0 kudos

Retrieved Docs in Message History

Hi,I've deployed a Langchain model via MLFlow log model and a deployment agent (following this https://notebooks.databricks.com/demos/llm-rag-chatbot/index.html#)How do I add additional content to the message history for example, I'd like to add retr...

  • 4471 Views
  • 2 replies
  • 0 kudos
Latest Reply
Brian-Nowak
Databricks Partner
  • 0 kudos

Hi @davidhuang thanks for pointing to that demo! That implementation of chat history did work in my experience, but it's a little simplistic and could be token-intensive if the history needs to be maintained as a list like that. Has Databricks done a...

  • 0 kudos
1 More Replies
GeorgeAristides
by • New Contributor III
  • 32487 Views
  • 5 replies
  • 7 kudos

Unity Catalog

I am new to the Databricks platform, and I am following the LLM course on youtube after being captivated by the excellent conceptual explanations and a longstanding desire to use this platform.Running the first cell in the module -%run ../Includes/Cl...

  • 32487 Views
  • 5 replies
  • 7 kudos
Latest Reply
ishmaelamin
Databricks Partner
  • 7 kudos

I was able to address this issue by upgrading the pip library, next I modified subsequent code that attempted to update specific versions of libraries needed by the code by simply importing the specific versions.  

  • 7 kudos
4 More Replies
cleversuresh
by • New Contributor III
  • 5218 Views
  • 3 replies
  • 5 kudos

Generative Certification notebook error

I am preparing for Generative Certification and facing the below error (LookupError:) when running the notebook in the first module. These were running without any errors in the past.Complete error stack trace:****************************************...

  • 5218 Views
  • 3 replies
  • 5 kudos
Latest Reply
MohammedArif
Databricks Partner
  • 5 kudos

Define all the helper functions from include folder (_helper_functions.py) in your main notebook itself and run these:import nltknltk.download('punkt')nltk.download('wordnet')nltk.download('omw-1.4')nltk.download('punkt_tab')nltk.download('averaged_p...

  • 5 kudos
2 More Replies
MauricioLins
by • Databricks Partner
  • 4083 Views
  • 1 replies
  • 0 kudos

Generative Certification notebook error

Hello,I have been trying to run the GenAI course notebook and this error below. I've already tried to download the averaged_perceptron_tagger_eng as it suggests, but the error keeps coming and I am running out of ideas, does anyone can help please?Re...

  • 4083 Views
  • 1 replies
  • 0 kudos
Latest Reply
MohammedArif
Databricks Partner
  • 0 kudos

Hi,I too was getting same error while running the notebook on my cluster. Got fixed after running these. import nltknltk.download('punkt')nltk.download('wordnet')nltk.download('omw-1.4')nltk.download('punkt_tab')nltk.download('averaged_perceptron_tag...

  • 0 kudos
koantek_user
by • Databricks Partner
  • 10003 Views
  • 1 replies
  • 1 kudos

What model is powering the AI/BI Genie in Databricks

Hello,We are conducting feasibility study of securely opening up the new features to our team members and want to use AI/BI Genie  - Genie Spaces1. How does it leverage the AI underneath .. what AI service is used , is the data shared with AI service...

  • 10003 Views
  • 1 replies
  • 1 kudos
Latest Reply
jamesl
Databricks Employee
  • 1 kudos

Hi @koantek_user, In case you haven't already found answers in documentation, AI/BI Genie uses Azure OpenAI -- your data is not used for training and model partners do not retain your data. (see trust and safety) Your data is governed and secured by ...

  • 1 kudos
Srav
by • New Contributor
  • 11536 Views
  • 2 replies
  • 0 kudos

I have not received the badge for Generative AI Fundamentals Accreditation

Hi Team,I have completed the course and cleared the exam of Generative AI Fundamentals Accreditation, but I did not receive the related badges yet. It's been 3 days.Could you please look into this matter?I appreciate if you can help.Thanks,Sravanthi

  • 11536 Views
  • 2 replies
  • 0 kudos
Latest Reply
Juan_Cardona
Databricks Partner
  • 0 kudos

Hi srav, I have been on the same position, the badges usually arrives on 1 or 3 days, but there are some cases that they take a little more time to arrive, maybe 1 or 2 weeks. 

  • 0 kudos
1 More Replies
nagND
by • Databricks Partner
  • 9113 Views
  • 1 replies
  • 2 kudos

Resolved! Mosaic Vector Search

I created a RAG using a corpus of pdf which I have on ADLS. Now where will the chunked text and vector embedding be stored once I parse all the PDFs and I want to start retrieval?  

  • 9113 Views
  • 1 replies
  • 2 kudos
Latest Reply
LauJohansson
Databricks Partner
  • 2 kudos

Option 1: Delta Sync Index with embeddings computed by Databricks You provide a source Delta table that contains data in text format. Databricks calculates the embeddings, using a model that you specify, and optionally saves the embeddings to a table...

  • 2 kudos
Cummings012
by • New Contributor II
  • 2199 Views
  • 1 replies
  • 1 kudos

Is there a batch query option for similar vectors available in Mosaic AI Vector Search?

Hi all,I have been exploring databrick's vector search offering, and was wondering if it had the ability to run a batch query?I would like to be able to do something like the following where both embedding_1 and embedding_2 are separate vectors that ...

Cummings012_0-1729087701912.png
  • 2199 Views
  • 1 replies
  • 1 kudos
Latest Reply
filipniziol
Esteemed Contributor
  • 1 kudos

Hi @Cummings012 ,As per documentation it is not possible, as the API supports single queries.The workaround is to process these queries in parallel to improve throughput.

  • 1 kudos
swollazn
by • Databricks Partner
  • 3774 Views
  • 3 replies
  • 0 kudos

Pyfunc mlflow loading Databricks Model failing

I am running this example notebook https://docs.databricks.com/_extras/notebooks/source/machine-learning/large-language-models/pyspark-dataframe-loader-langchain.htmlI am running most as if except switching embeddings and llm to databricks from OpenA...

  • 3774 Views
  • 3 replies
  • 0 kudos
Latest Reply
swollazn
Databricks Partner
  • 0 kudos

We abandoned this. Too many compatibility errors and we stuck with the notebook level. Good luck!

  • 0 kudos
2 More Replies
WynanddB
by • Databricks Partner
  • 4380 Views
  • 4 replies
  • 2 kudos

Enabling vector search

I have a team member doing the GenAI certification but she gets the following errorResponse content b'{"error_code":"BAD_REQUEST","message":"This trial workspace no longer has access to Vector Search. Please upgrade the workspace to continue to use V...

  • 4380 Views
  • 4 replies
  • 2 kudos
Latest Reply
nagND
Databricks Partner
  • 2 kudos

I am trying to enable mosiac vector search for my workspace but so far I can see the metastore in my catalog but no vector search.though I am on paid tier and have enabled unity catalog on the workspace. I am still not able to see vector search on my...

  • 2 kudos
3 More Replies
dinusr
by • New Contributor
  • 6746 Views
  • 1 replies
  • 0 kudos

from langchain.qa import RetrievalQA

Hi ,I have installed the library using "%pip install langchain", but still I couldn't import RetrievalQA. Could anyone help?

  • 6746 Views
  • 1 replies
  • 0 kudos
Latest Reply
filipniziol
Esteemed Contributor
  • 0 kudos

Hi @dinusr ,Are you using the correct namespace langchain.chains to import?from langchain.chains import RetrievalQAIf yes, try to 1. Install the most recent version %pip install --upgrade --force-reinstall langchain2. Restart the kernel : dbutils.lib...

  • 0 kudos
Galactech
by • New Contributor III
  • 5492 Views
  • 1 replies
  • 0 kudos

Resolved! Vector Index Creation Initializing Phase

When using the web UI to create a vector index from an existing table with chunked data the creation fails at the "Initializing" phase after about 20 minutes.  I have validated the requirements are satisfied that are detailed here.I have also seen th...

  • 5492 Views
  • 1 replies
  • 0 kudos
Latest Reply
Galactech
New Contributor III
  • 0 kudos

I believe this "self-resolved".  Even though I was technically on a premium plan the trial period had not completed.  I think at this point I can say it is resolved.

  • 0 kudos
Labels