By using the lateral and joining to your table you are exploding out the data to a point it will struggle to handle it. I would consider breaking your table and vector search index in into managable chunks and processing in stages. Usually you know you don't need to join everything to everything to search. You may also want to consider not doing it in SQL.