How to fully enable row-level concurrency on Databricks 14.1

MosefJJunior3rd — Fri, 20 Sep 2024 02:19:43 GMT

Hey guys,

I hope whoever's reading this is doing well.

We're trying to enable row-level concurrency on Databricks 14.1. However, the documentation seems a bit contradictory as to whether this is possible, and whether the whole capability of row-level concurrency is available. Doco here: https://docs.databricks.com/en/optimizations/isolation-level.html

At the top of the doco, it says,

Row-level concurrency is generally available on Databricks Runtime 14.2 and above,

and all the way at the bottom it says,

In Databricks Runtime 14.0 and 14.1, you can enable row-level concurrency for tables with deletion vectors,

but it also says,

In Databricks Runtime 14.1 and below, non-Photon compute only supports row-level concurrency for DELETE operations.

From what I've put together, this means that low-level concurrency is only available on Databricks Runtime 14.1 only when Photon is enabled, and when deletion-vectors are enabled on the particular table, and row-level concurrency must be explicitly enabled in the cluster configuration. These three conditions must be met simultaneously to enable and unlock the capabilities of row-level concurrency.

Can I just get some confirmation from someone who's managed to do this before? Much appreciated, thank you.

Regards,

Re: How to fully enable row-level concurrency on Databricks 14.1

gchandra — Tue, 24 Sep 2024 02:06:48 GMT

Yes, your understanding is correct. As best practice, use 14.3 LTS DBR, as LTS has longer support compared to 6 months.

topic How to fully enable row-level concurrency on Databricks 14.1 in Administration & Architecture

How to fully enable row-level concurrency on Databricks 14.1

Re: How to fully enable row-level concurrency on Databricks 14.1