Roadmap for vector_search function

jAAmes_bentley — Wed, 16 Apr 2025 09:22:55 GMT

I was wondering if there was a roadmap for the development of the vector_search function: vector_search function | Databricks Documentation

Specifically, I was wondering if / when the following limitations may be lifted:

Querying DIRECT_ACCESS index types are not supported.
Input parameters filters_json are not supported.
Hybrid keyword-similarity search is not supported using vector_search().

We are currently developing a solution which requires the use of a direct access vector store and batch similarity searches, in which we require the use of filters and preferably hybrid search (though hybrid search is more of a nice to have for now).

Currently our solution uses an async function to do this, but this takes a while:

from databricks.vector_search.client import VectorSearchClient index.similarity_search( query_text= q, num_results= k, query_vector= embedded_query, filters= {"document_id": document_id_filter}, query_type= "hybrid" )

It would be great to know when we may be able to switch to using Databricks SQL vector_search? We have noticed significant performance increases from switching to ai_query and it would also be nice to keep everything in spark dataframe format throughout the process.

Thanks!

Re: Roadmap for vector_search function

Vinay_M_R — Tue, 29 Apr 2025 04:50:32 GMT

Hello @jAAmes_bentley,

DIRECT_ACCESS & filters_json are not currently supported with vector_search sql function. These are on our roadmap, but we don’t have concrete ETAs to share at the moment as we’re focusing on other high-priority tasks.

Hybrid search is currently being rolled out.

topic Re: Roadmap for vector_search function in Generative AI

Roadmap for vector_search function

Re: Roadmap for vector_search function