cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Generative AI
Explore discussions on generative artificial intelligence techniques and applications within the Databricks Community. Share ideas, challenges, and breakthroughs in this cutting-edge field.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Vector Search Indexes on dlt pipelines / materialized views

dbx_beginner_10
New Contributor II

Hello,

 

I am not seeing the "create vector search index" option on my materialized views I created from Databricks Asset Bundles using dlt.

 

Is this expected behavior?

2 REPLIES 2

nayan_wylde
Honored Contributor

 DLT materialized views have different properties than regular Delta tables, and vector search indexing may not be directly supported on them through the UI.

Try creating programatically.

client = VectorSearchClient()

# Create the index
client.create_delta_sync_index(
    endpoint_name="your-endpoint-name",
    index_name="your-index-name",
    source_table_name="your.dlt.materialized_view",
    pipeline_type="TRIGGERED",  # or "CONTINUOUS"
    primary_key="id",
    embedding_source_column="text_column",
    embedding_model_endpoint_name="your-embedding-endpoint"
)

dbx_beginner_10
New Contributor II

Yes, that failed because I didn't have change data capture on, which I learned materialized views don't support.

I really like the dev/prod Databricks Asset Bundles, but I'm not sure the forcing of materialized views is compatible with a lot of work I need to do?