Thank you for taking the time to answer my question in such detail. The link to the delta protocol file (PROTOCOL.md) on GitHub should be of great help to anyone looking to understand how the _delta_log folder is utilized.

What if individual log files have been permanently deleted but log compaction files are still available?

I had another table whose log files had been permanently deleted, so I couldn't recover them by undeleting soft deletes. There was no gap in the log files: each version was covered by either a checkpoint, a log compaction file, or an individual log file.

I found a hack that allowed me to recover the table. The hack was to create missing log files without any content.

dbutils.fs.put("abfss://container@storage_account/delta_table/path/_delta_log/00000000000000000959.json", "")

I was able to guess the hack because a daily job read the table every day, but the files were still deleted by the data lifecycle policy. This meant the job probably didn't look inside the files. If it did, the files would be "fresh" to the lifecycle policy. Since I only cared about the latest state of the table, empty files did the trick. I did not have to change the DBR version.

Warning: I'm unaware if there are any side effects to this trick (besides a broken time travel).