Multi Vector Index
Can we create multi vector indexes on multiple embedding columns in delta table as part of databricks vector search
- 241 Views
- 0 replies
- 0 kudos
Can we create multi vector indexes on multiple embedding columns in delta table as part of databricks vector search
Does Databricks vector search support pre-filtering or post-filtering or both in metadata filtering?
Hello @SaiVihAI, Yes, It uses in-query filtering. The algorithm takes a series of iterative steps to determine if the filtered set is small enough to execute brute-force search or if it proceeds with AN
Regarding copying or cloning vector index tables, it appears that direct copying or cloning is not possible, as Databricks throws the message: "Securable with kind TABLE_ONLINE_VECTOR_INDEX_REPLICA does not support Lakehouse Federation."It seems that...
Direct cloning of a vector index table may not be possible. However, if Databricks generates the embeddings, you can enable "Sync computed embeddings" to save them to a Unity Catalog table. Alternatively, you can set the parameter sync_computed_embed...
Hi Team,I would like to use Genie to enable NLP on my delta lake. However, the documentation mention the limitation as 20 messages per minute for a workspace for all Genie Spaces.Is there a pay-as-go model for Genie Space?Regards,Ravi Kumar Singh
Hi everyone,I’m currently working with the new Synthetic Data Generation API in Mosaic AI, and I’ve encountered an issue when trying to generate questions for a document. Specifically, I’m running into the following error:Failed to generate question...
Hi @aero_g, Looks like it is not supported on your region, but I will get more details internally.
Hi Team,I am following below article to evaluate my model.What is Mosaic AI Agent Evaluation? | Databricks on AWSExample for my model is below:input_example = pd.DataFrame({ "text": ["What is the stoppage duration for January month 2024?"], "pa...
Hello @Nawneet, Ensure that the input format for the agent evaluation matches the expected schema. Here is an example to structure your input data correctly: import mlflow import pandas as pd examples = { "request": [ {"messages": [{"ro...
When I try to use Datbricks sql agent, I'm getting below error:DatabaseError: (databricks.sql.exc.ServerOperationError) [UNBOUND_SQL_PARAMETER] Found the unbound parameter: param_1. Please, fix `args` and provide a mapping of the parameter to a SQL ...
Hi @dileepkumar_t The UNBOUND_SQL_PARAMETER error in Databricks SQL Agent occurs when a parameter marker in a SQL query is not associated with a value. This error is raised because the SQL query is expecting a parameter to be provided, but it has not...
Hello,I am implementing RAG solution as per Databricks cookbook. Review App is working, references are provided as text chunks.I need to build functionality to open pdf file on a specific page as a reference. Is there a way to change ReviewApp to ope...
I am facing issues calling the Llama Guard Model after deploying it to Model Serving on Databricks. I deployed the model by requesting instant access from Databricks Marketplace and deployed it to Model Serving.I am following the code examples here:...
@javieryw, The error you are encountering, "400 Client Error: Bad Request for url: https://westus.azuredatabricks.net/serving-endpoints/llama-guard/invocations. Response text: Bad request: json: unknown field 'dataframe_split'", indicates that the p...
Hi all.I would like to embed Genie Space in my company's data intranet to be used like a "ChatGPT", would it be possible?Or use an API to make a call from Genie Space (in this case we would develop an interface on the intranet and send the questions ...
@felipe_silva Have you got api for the same
I created a RAG application in databricks which performs the following steps:1. Extract text from PDF files2. Prepare embeddings on extracted text and create vector search index3. Create a LLM model and served the model which can answer question base...
How to unlock enterprise AI: “You can have all the AI in the world, but if it’s on a shaky data foundation, then it’s not going to bring you any value.” — Carol Clements, Chief Digital and Technology Officer, JetBlue Companies everywhere have been qu...
Hello everyone,I’m working on a translation project involving documents up to 100 pages long, in 17 different languages, and I'm looking for the best approach to achieve high-quality translations in this multilingual context.Single model vs. multi-mo...
Hello,We are conducting feasibility study of securely opening up the new features to our team members and want to use AI/BI Genie - Genie Spaces1. How does it leverage the AI underneath .. what AI service is used , is the data shared with AI service...
Hi @koantek_user, In case you haven't already found answers in documentation, AI/BI Genie uses Azure OpenAI -- your data is not used for training and model partners do not retain your data. (see trust and safety) Your data is governed and secured by ...
Hi ,I have installed the library using "%pip install langchain", but still I couldn't import RetrievalQA. Could anyone help?
Hi @dinusr ,Are you using the correct namespace langchain.chains to import?from langchain.chains import RetrievalQAIf yes, try to 1. Install the most recent version %pip install --upgrade --force-reinstall langchain2. Restart the kernel : dbutils.lib...