- 30842 Views
- 19 replies
- 34 kudos
Databricks Announces the Industry’s First Generative AI Engineer Learning Pathway and Certification
Today, we are announcing the industry's first Generative AI Engineer learning pathway and certification to help ensure that data and AI practitioners have the resources to be successful with generative AI. At Databricks, we recognize that generative ...
- 30842 Views
- 19 replies
- 34 kudos
- 34 kudos
This is an exciting step forward from Databricks! Looking forward to diving into the curriculum and exploring what's next in the world of data + AI! Thanks for sharing @Sujitha
- 34 kudos
- 3348 Views
- 1 replies
- 0 kudos
Resolved! Roadmap for vector_search function
I was wondering if there was a roadmap for the development of the vector_search function: vector_search function | Databricks DocumentationSpecifically, I was wondering if / when the following limitations may be lifted:Querying DIRECT_ACCESS index ty...
- 3348 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @jAAmes_bentley, DIRECT_ACCESS & filters_json are not currently supported with vector_search sql function. These are on our roadmap, but we don’t have concrete ETAs to share at the moment as we’re focusing on other high-priority tasks. Hybrid s...
- 0 kudos
- 7434 Views
- 1 replies
- 0 kudos
Resolved! How to do NLP against PDFs in Databricks? Can be done in Snowflake very easily.
I’d love to build a quick "art of the possible" demo showing how easy it is to query unstructured PDFs using natural language. In Snowflake, I wired up a similar solution in ~2 hours just by following their tutorial guide.Does anyone know the best wa...
- 7434 Views
- 1 replies
- 0 kudos
- 0 kudos
To query unstructured PDF files using natural language in Databricks, you can leverage an approach similar to the "Retrieval Augmented Generation (RAG) and DBRX" demo. Although the specific demo you referenced (https://notebooks.databricks.com/demos/...
- 0 kudos
- 2512 Views
- 4 replies
- 3 kudos
LangChain Compatibility in Databricks – Painfully Inconsistent Experience
Hey all,Is anyone else experiencing major friction when using LangChain within Databricks?Even after carefully pinning versions like:%pip install \ loguru \ mlflow==2.10.1 \ langchain==0.1.5 \ langchain-google-vertexai \ databricks-vectorse...
- 2512 Views
- 4 replies
- 3 kudos
- 3 kudos
I'll definitely look into installing libraries at the cluster level via init scripts makes a lot of sense for stability and reproducibility. I hadn't considered that approach, so I really appreciate you highlighting that and providing the relevant Da...
- 3 kudos
- 3592 Views
- 1 replies
- 0 kudos
Getting "The given endpoint does not exist" for LLM endpoint though I've access to it
When trying out this LLM RAG notebook steps I'm getting this error:"The given endpoint does not exist, please retry after checking the specified model and version deployment exists"This notebook originally referred to databricks-meta-llama-3-1-70b-in...
- 3592 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @Iris12 ,Not sure if this will fix your issue, but I’ve seen similar errors caused by permission mismatches.Make sure that the user or service principal you’re using has the correct permissions both in the Models and Serving tab.Also, double-chec...
- 0 kudos
- 14633 Views
- 5 replies
- 2 kudos
Genie Space Embedded or access using a REST API
Hi all.I would like to embed Genie Space in my company's data intranet to be used like a "ChatGPT", would it be possible?Or use an API to make a call from Genie Space (in this case we would develop an interface on the intranet and send the questions ...
- 14633 Views
- 5 replies
- 2 kudos
- 2 kudos
Hi all,I just came across this thread and wanted to jump in — I’m currently working on implementing a Genie Space that will be used by external customers, and we’re also looking into whether it can be accessed via API or embedded into a custom fronte...
- 2 kudos
- 1046 Views
- 1 replies
- 0 kudos
Genie - Data Lineage
Would it be possible for Genie to bring in the Lineage Graph of table?Thanks
- 1046 Views
- 1 replies
- 0 kudos
- 0 kudos
Have you tried looking at lineage through the system tables: "system.access.table_lineage" and "system.access.column_lineage". You would need to have entitlements on these objects but you should be able to reference them in Genie. Give it a try and...
- 0 kudos
- 1737 Views
- 2 replies
- 0 kudos
Issue with Lang Chain Agents
This is regarding Agentic framework. We are getting issues while invoking the lang chain agent and need your help to get some suggestions from your expert team to resolve this.As per our observation Agent is behaving bit weird while calling tools reg...
- 1737 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @SandipCoder, About point 2, is it working intermittently or failing all the time?
- 0 kudos
- 2550 Views
- 2 replies
- 0 kudos
How to deploy an agent into model registry using MLFlow
Goal :Here below I am attaching my code snippet where I am trying to build an agent on my custom functions .Things are working well when I am running agent.invoke(“Question”).But now I want to deploy this agent into model registry using MLFlow but I ...
- 2550 Views
- 2 replies
- 0 kudos
- 0 kudos
Had the same question when I was trying to deploy my custom llm. How we did it was wrapping it inside a custom mlflow model and registering it.
- 0 kudos
- 16950 Views
- 1 replies
- 5 kudos
Databricks MCP Server
Hey, couldn't find a working Databricks MCP so I forked one and fixed it: https://github.com/markov-kernel/databricks-mcp This allows you to (using Cursor, Claude Code, Windsurf, Claude, etc.):* Manage Databricks Clusters: List, create, terminate, ...
- 16950 Views
- 1 replies
- 5 kudos
- 5 kudos
very cool
- 5 kudos
- 930 Views
- 1 replies
- 0 kudos
Datbricks Notebook as a Server ?
Hello, What I'm trying to do : I have a piece of python code (an ai application) that i want to deploy as a server, almost exactly like one would lets say on an EC2. The only difference being instead of a flask api i would use dbutils and the databri...
- 930 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Krthk, Thanks for your question. Have you thought about using Model serving endpoint for your use-case: https://docs.databricks.com/aws/en/machine-learning/model-serving/ Launching a Databricks job via the REST API and dbutils is a different para...
- 0 kudos
- 3562 Views
- 3 replies
- 1 kudos
Resolved! Custom LLM Function similar to Databricks built in AI functions
Hi Team,I am new to Databricks and working on Gen AI requirements.Task - Build custom prompt with set of instructions to classify incident e.g. False Positive or True Positive. Input will be columns in Delta TableCan I build custom LLM function with ...
- 3562 Views
- 3 replies
- 1 kudos
- 4220 Views
- 1 replies
- 0 kudos
Multi Vector Index
Can we create multi vector indexes on multiple embedding columns in delta table as part of databricks vector search
- 4220 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi SaiVihAIDatabricks Vector Search currently does not support creating multi-vector indexes that are directly based on multiple embedding columns in a Delta table. Creating separate vector indexes for each embedding column is an option.Ref How to c...
- 0 kudos
- 1520 Views
- 1 replies
- 0 kudos
Vector Quantization
Does Databricks Vector Search support vector quantization?
- 1520 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi SaiVihAIIn my opinion, Databricks utilizes Mosaic Vector Search, which is largely aided by the Hierarchial Navigable Small World (HNSW) algorithm for neighbor search and hybrid keyword similarity search.Ref Mosaic AI Vector Search | Databricks D...
- 0 kudos
- 5906 Views
- 7 replies
- 5 kudos
Genie Agent Integration in Databricks
I'm developing an Agent using the Cookbook template and would like to incorporate a Genie agent. I came across the Databricks GitHub repository and noticed they're actively working on it: GitHub - Genie Agent. However, it seems to be in the early sta...
- 5906 Views
- 7 replies
- 5 kudos
- 5 kudos
I think Teams/Slack channel as a chat bot will be more exciting use case. When we demoed the Genie to our executives we are in Retail Industry Client and we have lots of stores one of the thing they said is this would be awesome if this can be in Tea...
- 5 kudos
- 7604 Views
- 4 replies
- 0 kudos
Resolved! LangGraph MemorySaver checkpointer usage with MLflow
Hi everyone.I am working on a graph that utilizes the MemorySaver class to incorporate short-term memory. This will enable me to maintain a multi-turn conversation with the user by storing the chat history.I am using the MLflow "models from code" fea...
- 7604 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @moemedina. No, I didn't.I'm considering using ChatModel/ChatAgent class to wrap the graph and be able to move on. However, the MLflow documentation is still referring to ChatModel where Chat Agent is the latest recommendation:MLflow ChatModel Doc...
- 0 kudos
