How to fully enable row-level concurrency on Databricks 14.1
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ09-19-2024 07:19 PM
Hey guys,
I hope whoever's reading this is doing well.
We're trying to enable row-level concurrency on Databricks 14.1. However, the documentation seems a bit contradictory as to whether this is possible, and whether the whole capability of row-level concurrency is available. Doco here: https://docs.databricks.com/en/optimizations/isolation-level.html
At the top of the doco, it says,
Row-level concurrency is generally available on Databricks Runtime 14.2 and above,
and all the way at the bottom it says,
In Databricks Runtime 14.0 and 14.1, you can enable row-level concurrency for tables with deletion vectors,
but it also says,
In Databricks Runtime 14.1 and below, non-Photon compute only supports row-level concurrency for DELETE operations.
From what I've put together, this means that low-level concurrency is only available on Databricks Runtime 14.1 only when Photon is enabled, and when deletion-vectors are enabled on the particular table, and row-level concurrency must be explicitly enabled in the cluster configuration. These three conditions must be met simultaneously to enable and unlock the capabilities of row-level concurrency.
Can I just get some confirmation from someone who's managed to do this before? Much appreciated, thank you.
Regards,
Mo
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
โ09-23-2024 07:06 PM
Yes, your understanding is correct. As best practice, use 14.3 LTS DBR, as LTS has longer support compared to 6 months.
~

