cancel
Showing results for 
Search instead for 
Did you mean: 
Generative AI
Explore discussions on generative artificial intelligence techniques and applications within the Databricks Community. Share ideas, challenges, and breakthroughs in this cutting-edge field.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Sujitha
by Databricks Employee
  • 34196 Views
  • 22 replies
  • 46 kudos

Databricks Announces the Industry’s First Generative AI Engineer Learning Pathway and Certification

Today, we are announcing the industry's first Generative AI Engineer learning pathway and certification to help ensure that data and AI practitioners have the resources to be successful with generative AI. At Databricks, we recognize that generative ...

Screenshot 2024-01-24 at 11.32.01 PM.png
  • 34196 Views
  • 22 replies
  • 46 kudos
Latest Reply
Darshan137
New Contributor II
  • 46 kudos

Dear Certifications TeamI have completed full Generative AI Engineering Pathway, so I received module wise knowledge badge but I didn't received the overall certificate which mentioned in description which is Generative AI Engineer with one Star. Req...

  • 46 kudos
21 More Replies
Iris12
by New Contributor
  • 4256 Views
  • 1 replies
  • 0 kudos

Getting "The given endpoint does not exist" for LLM endpoint though I've access to it

When trying out this LLM RAG notebook steps I'm getting this error:"The given endpoint does not exist, please retry after checking the specified model and version deployment exists"This notebook originally referred to databricks-meta-llama-3-1-70b-in...

  • 4256 Views
  • 1 replies
  • 0 kudos
Latest Reply
Isi
Honored Contributor III
  • 0 kudos

Hey @Iris12 ,Not sure if this will fix your issue, but I’ve seen similar errors caused by permission mismatches.Make sure that the user or service principal you’re using has the correct permissions both in the Models and Serving tab.Also, double-chec...

  • 0 kudos
felipe_silva
by New Contributor II
  • 15741 Views
  • 5 replies
  • 2 kudos

Genie Space Embedded or access using a REST API

Hi all.I would like to embed Genie Space in my company's data intranet to be used like a "ChatGPT", would it be possible?Or use an API to make a call from Genie Space (in this case we would develop an interface on the intranet and send the questions ...

  • 15741 Views
  • 5 replies
  • 2 kudos
Latest Reply
mark_dingli
New Contributor II
  • 2 kudos

Hi all,I just came across this thread and wanted to jump in — I’m currently working on implementing a Genie Space that will be used by external customers, and we’re also looking into whether it can be accessed via API or embedded into a custom fronte...

  • 2 kudos
4 More Replies
rjhari
by New Contributor
  • 1322 Views
  • 1 replies
  • 0 kudos

Genie - Data Lineage

Would it be possible for Genie to bring in the Lineage Graph of table?Thanks

  • 1322 Views
  • 1 replies
  • 0 kudos
Latest Reply
Louis_Frolio
Databricks Employee
  • 0 kudos

Have you tried looking at lineage through the system tables: "system.access.table_lineage" and "system.access.column_lineage".  You would need to have entitlements on these objects but you should be able to reference them in Genie.  Give it a try and...

  • 0 kudos
SandipCoder
by New Contributor II
  • 2648 Views
  • 2 replies
  • 0 kudos

Issue with Lang Chain Agents

This is regarding Agentic framework. We are getting issues while invoking the lang chain agent and need your help to get some suggestions from your expert team to resolve this.As per our observation Agent is behaving bit weird while calling tools reg...

SandipCoder_0-1736512522115.png SandipCoder_1-1736512522122.png
  • 2648 Views
  • 2 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @SandipCoder, About point 2, is it working intermittently or failing all the time?

  • 0 kudos
1 More Replies
SandipCoder
by New Contributor II
  • 3147 Views
  • 2 replies
  • 0 kudos

How to deploy an agent into model registry using MLFlow

Goal :Here below I am attaching my code snippet where I am trying to build an agent on my custom functions .Things are working well when I am running agent.invoke(“Question”).But now I want to deploy this agent into model registry using MLFlow but I ...

  • 3147 Views
  • 2 replies
  • 0 kudos
Latest Reply
prar_shah
New Contributor III
  • 0 kudos

Had the same question when I was trying to deploy my custom llm. How we did it was wrapping it inside a custom mlflow model and registering it.

  • 0 kudos
1 More Replies
olivierddr
by New Contributor III
  • 18076 Views
  • 1 replies
  • 6 kudos

Databricks MCP Server

Hey, couldn't find a working Databricks MCP so I forked one and fixed it: https://github.com/markov-kernel/databricks-mcp This allows you to (using Cursor, Claude Code, Windsurf, Claude, etc.):*   Manage Databricks Clusters: List, create, terminate, ...

  • 18076 Views
  • 1 replies
  • 6 kudos
Latest Reply
spaceghost
New Contributor II
  • 6 kudos

very cool

  • 6 kudos
Krthk
by New Contributor
  • 1326 Views
  • 1 replies
  • 0 kudos

Datbricks Notebook as a Server ?

Hello, What I'm trying to do : I have a piece of python code (an ai application) that i want to deploy as a server, almost exactly like one would lets say on an EC2. The only difference being instead of a flask api i would use dbutils and the databri...

  • 1326 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @Krthk, Thanks for your question. Have you thought about using Model serving endpoint for your use-case: https://docs.databricks.com/aws/en/machine-learning/model-serving/ Launching a Databricks job via the REST API and dbutils is a different para...

  • 0 kudos
neha89
by New Contributor II
  • 4061 Views
  • 3 replies
  • 1 kudos

Resolved! Custom LLM Function similar to Databricks built in AI functions

Hi Team,I am new to Databricks and working on Gen AI requirements.Task - Build custom prompt with set of instructions to classify incident e.g. False Positive or True Positive. Input will be columns in Delta TableCan I build custom LLM function with ...

  • 4061 Views
  • 3 replies
  • 1 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 1 kudos

No problem, if you have any other question, let me know!

  • 1 kudos
2 More Replies
SaiVihAI
by New Contributor
  • 1782 Views
  • 1 replies
  • 0 kudos

Vector Quantization

Does Databricks Vector Search support vector quantization?

  • 1782 Views
  • 1 replies
  • 0 kudos
Latest Reply
santhakumar11
New Contributor III
  • 0 kudos

Hi SaiVihAIIn my opinion, Databricks utilizes Mosaic Vector Search, which is largely aided by the Hierarchial Navigable Small World (HNSW) algorithm for neighbor search and hybrid keyword similarity search.Ref   Mosaic AI Vector Search | Databricks D...

  • 0 kudos
aliciachimeno
by New Contributor III
  • 7747 Views
  • 7 replies
  • 5 kudos

Genie Agent Integration in Databricks

I'm developing an Agent using the Cookbook template and would like to incorporate a Genie agent. I came across the Databricks GitHub repository and noticed they're actively working on it: GitHub - Genie Agent. However, it seems to be in the early sta...

  • 7747 Views
  • 7 replies
  • 5 kudos
Latest Reply
NathanSundarara
Valued Contributor
  • 5 kudos

I think Teams/Slack channel as a chat bot will be more exciting use case. When we demoed the Genie to our executives we are in Retail Industry Client and we have lots of stores one of the thing they said is this would be awesome if this can be in Tea...

  • 5 kudos
6 More Replies
sebascardonal
by Databricks Partner
  • 14441 Views
  • 4 replies
  • 0 kudos

Resolved! LangGraph MemorySaver checkpointer usage with MLflow

Hi everyone.I am working on a graph that utilizes the MemorySaver class to incorporate short-term memory. This will enable me to maintain a multi-turn conversation with the user by storing the chat history.I am using the MLflow "models from code" fea...

  • 14441 Views
  • 4 replies
  • 0 kudos
Latest Reply
sebascardonal
Databricks Partner
  • 0 kudos

Hi @moemedina. No, I didn't.I'm considering using ChatModel/ChatAgent class to wrap the graph and be able to move on. However, the MLflow documentation is still referring to ChatModel where Chat Agent is the latest recommendation:MLflow ChatModel Doc...

  • 0 kudos
3 More Replies
SaiVihAI
by New Contributor
  • 1546 Views
  • 1 replies
  • 0 kudos

Metadata filtering mechanism in Databricks Vector Search

Does Databricks vector search support pre-filtering or post-filtering or both in metadata filtering?

  • 1546 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hello @SaiVihAI, Yes, It uses in-query filtering. The algorithm takes a series of iterative steps to determine if the filtered set is small enough to execute brute-force search or if it proceeds with AN

  • 0 kudos
Mohamednazeer
by New Contributor III
  • 7059 Views
  • 2 replies
  • 2 kudos

Serving endpoint with external model (azure openai) is throwing "Public network access is disable "

Databricks serving endpoint is not working as expected throwing an exception as "Public network access is disable, Create private endpoints".We have create the azure openAI resource with public network access disabled, but also we have created the pr...

  • 7059 Views
  • 2 replies
  • 2 kudos
Latest Reply
Ismael-K
Databricks Employee
  • 2 kudos

Serving endpoints with external models will require creating Network Connectivity Configuration objects at the account level to connect to the Azure OpenAI resource. The feature is currently in private preview, please reach out to your Databricks acc...

  • 2 kudos
1 More Replies
VELU1122
by New Contributor II
  • 5686 Views
  • 1 replies
  • 0 kudos

Copying/cloning Vector index table

Regarding copying or cloning vector index tables, it appears that direct copying or cloning is not possible, as Databricks throws the message: "Securable with kind TABLE_ONLINE_VECTOR_INDEX_REPLICA does not support Lakehouse Federation."It seems that...

  • 5686 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ismael-K
Databricks Employee
  • 0 kudos

Direct cloning of a vector index table may not be possible. However, if Databricks generates the embeddings, you can enable "Sync computed embeddings" to save them to a Unity Catalog table. Alternatively, you can set the parameter sync_computed_embed...

  • 0 kudos
Paolo_Finardi
by New Contributor III
  • 3559 Views
  • 4 replies
  • 2 kudos

Building a Chatbot Assistant for Data Analysts Leveraging Unity Catalog

Hey everyone,I'm working on a project to enhance our data analysts' experience on the platform. In our company, we've done a great job with Unity Catalog, and we have comprehensive descriptions for all the tables and columns in our data lake.I'm inte...

  • 3559 Views
  • 4 replies
  • 2 kudos
Latest Reply
Paolo_Finardi
New Contributor III
  • 2 kudos

Hi MariuszK,I can confirm that the RAG architecture on Databricks works very well—we’ve taken it to production and are very satisfied with the results. As a first step, I'll try creating a Genie Room on the system table to see if it can help retrieve...

  • 2 kudos
3 More Replies