Today, we are announcing the industry's first Generative AI Engineer learning pathway and certification to help ensure that data and AI practitioners have the resources to be successful with generative AI. At Databricks, we recognize that generative ...
I am building a text-to-sql agent using Langchain API.I created a SQLDatabase using: db = SQLDatabase.from_databricks(catalog="`my-catalog-name`", schema="my_schema", host="...", api_token="...", warehouse_id="12345678")When I call db.get_table_info(...
Hello, What I'm trying to do : I have a piece of python code (an ai application) that i want to deploy as a server, almost exactly like one would lets say on an EC2. The only difference being instead of a flask api i would use dbutils and the databri...
Hi @Krthk, Thanks for your question. Have you thought about using Model serving endpoint for your use-case: https://docs.databricks.com/aws/en/machine-learning/model-serving/ Launching a Databricks job via the REST API and dbutils is a different para...
Hi Team,I am new to Databricks and working on Gen AI requirements.Task - Build custom prompt with set of instructions to classify incident e.g. False Positive or True Positive. Input will be columns in Delta TableCan I build custom LLM function with ...
Can we create multi vector indexes on multiple embedding columns in delta table as part of databricks vector search
Hi SaiVihAIDatabricks Vector Search currently does not support creating multi-vector indexes that are directly based on multiple embedding columns in a Delta table. Creating separate vector indexes for each embedding column is an option.Ref How to c...
Does Databricks Vector Search support vector quantization?
Hi SaiVihAIIn my opinion, Databricks utilizes Mosaic Vector Search, which is largely aided by the Hierarchial Navigable Small World (HNSW) algorithm for neighbor search and hybrid keyword similarity search.Ref Mosaic AI Vector Search | Databricks D...
I'm developing an Agent using the Cookbook template and would like to incorporate a Genie agent. I came across the Databricks GitHub repository and noticed they're actively working on it: GitHub - Genie Agent. However, it seems to be in the early sta...
I think Teams/Slack channel as a chat bot will be more exciting use case. When we demoed the Genie to our executives we are in Retail Industry Client and we have lots of stores one of the thing they said is this would be awesome if this can be in Tea...
Hi, I'm building an streamlit application on databricks apps, where user can upload some data , and I run an LLM model and return results. There, I want to log an artifact to a volume. I'm following this documentation https://docs.databricks.com/aws...
How to integrate genie in with databricks - data apps i want a chatbot that behaves like genie
I'm working with vector search in Databricks using vector index sync with Delta tables, and I'm a bit unclear on how updates to the source table affect the vector index, specifically when using different write operations.If I overwrite the source Del...
Hello, I'm currently working with Databricks features to create an AI agent system. In the documentation it's written that the library DatabricksFunctionClient is needed to register custom tools to UC. I followed the different steps from the explana...
Hi everyone.I am working on a graph that utilizes the MemorySaver class to incorporate short-term memory. This will enable me to maintain a multi-turn conversation with the user by storing the chat history.I am using the MLflow "models from code" fea...
Hi @moemedina. No, I didn't.I'm considering using ChatModel/ChatAgent class to wrap the graph and be able to move on. However, the MLflow documentation is still referring to ChatModel where Chat Agent is the latest recommendation:MLflow ChatModel Doc...
Dear Databricks Community,I am seeking assistance with an issue I've encountered while deploying a model on Databricks. When invoking the serving endpoint, I receive the following error message sometimes:400 Client Error: Error: Model response did no...
Does Databricks Vector Search support Multi-Vector Embedding?
Does Databricks vector search support pre-filtering or post-filtering or both in metadata filtering?
Hello @SaiVihAI, Yes, It uses in-query filtering. The algorithm takes a series of iterative steps to determine if the filtered set is small enough to execute brute-force search or if it proceeds with AN
Hi! i would like to use my own sentence transformer to create a vector index. It is not a problem using mlflow sentence-transformer flavour, it works fine with: mlflow.sentence_transformers.log_model( model, artifact_path="model", signatu...
