Open Delta Sharing and Deletion Vectors

[DrK]

Hi,

Just experimenting with open delta sharing and running into a few technical traps.  Mainly that if deletion vectors are enabled on a delta table (which they are by default now) we get errors when trying to query a table (specifically with PowerBI).

i.e. "Table features delta.enableDeletionVectors are found in table version: 5. For DeletionVectors, use DBR with version 14.1(14.2 for CDF and streaming) or higher, or delta-sharing-spark with version 3.1 or higher, and set option (\"responseFormat\", \"delta\") to query the table. "

How do you effect the version of DBR (i.e. to use 14.2) what compute is delta-sharing using to be able to change the spec?  

[NandiniN]

Hi @DrK ,

From the error mentioned. 

If you try to set option("responseFormat", "delta") in the query, does it succeed?

To answer your question - In Databricks Runtime 14.2 and above, tables with deletion vectors support row-level concurrency. 

Databricks recommends using Databricks Runtime 14.3 LTS and above to write tables with deletion vectors to leverage all optimizations. 

[DrK]

Hi,

Thanks for the response, however, I don`t think you can set such parameters with the PowerBI native connector for delta, only parameter is a max row count.

[DarioBarbadillo]

I am also experiencing this issue when connecting through Power BI connector. @DrK did you manage to solve it?

[AndySkinner]

@DarioBarbadillo hi, not technically no, the only solution I`ve got is to remove the deletion vectors feature from the delta table!