Databricks Predictive I/O. With this new technology, we no longer have to rely on rewriting entire files when merging. Instead, deletion vectors identify and remove specific data points, leading to more efficient and streamlined processes. In addition, using deep learning enables Predictive I/O to determine the most optimal way to read and write data and automate feature extraction.
Since DBR 12.1, you enable support for deletion vectors on a Delta Lake table by setting a table property as shown below:
ALTER TABLE <table_name> SET TBLPROPERTIES ('delta.enableDeletionVectors' = true);
