databricks-vectorsearch 0.53 unable to use similarity_search()

snaveedgm — Fri, 21 Mar 2025 16:25:07 GMT

I have an issue with databricks-vectorsearch package. Version 0.51 suddenly stopped working this week because:

It now expected me to provide azure_tenant_id in addition to service principal's client ID and secret.
After supplying tenant ID, it showed some bug like "Unsupported Types" time.time() + oauth_token_data["expires_in"].
I then upgraded to 0.53 and this bug was resolved and some functions like get_index() started working. However, now the similarity_search() function gives:

b'{"error_code":"PERMISSION_DENIED","message":"Failed to call Model Serving endpoint: databricks-bge-large-en."}', status_code 403

I have tried querying the databricks-bge-large-en endpoint separately via REST for my service account and it works fine, so the issue seems specific to this package.

Please advise.

Code Snippet:

from databricks.vector_search.client import VectorSearchClient

vsc = VectorSearchClient(

workspace_url=workspace_url,

service_principal_client_id=sp_client_id,

service_principal_client_secret=sp_client_secret,

azure_tenant_id=tenant_id

)

index = vsc.get_index(endpoint_name=endpoint_name, index_name=index_name)

index.similarity_search(num_results=3, columns=["chunked_text"], query_text="example_query")

Re: databricks-vectorsearch 0.53 unable to use similarity_search()

stbjelcevic — Tue, 28 Oct 2025 15:58:31 GMT

Hi @snaveedgm ,

This is interesting - can you double-check that the service principal has CAN QUERY on the embedding endpoint used for ingestion and/or querying (databricks-bge-large-en in your case)? Even though your direct REST test works, double-check permissions are consistent across workspaces/endpoints if you have multi-workspace setups.

Given your environment and error, the fastest unblock is either PAT or query_vector:

from databricks.vector_search.client import VectorSearchClient # Use PAT (set DATABRICKS_TOKEN in the environment or pass personal_access_token) vsc = VectorSearchClient( workspace_url=workspace_url, personal_access_token=os.environ["DATABRICKS_TOKEN"] # or pass the token directly ) index = vsc.get_index(endpoint_name=endpoint_name, index_name=index_name)

topic Re: databricks-vectorsearch 0.53 unable to use similarity_search() in Machine Learning

databricks-vectorsearch 0.53 unable to use similarity_search()

Re: databricks-vectorsearch 0.53 unable to use similarity_search()