For what it's worth, we're using Databricks hosted in azure, and working through support, this problem did get resolved in the latest 12.2 version in that environment for us, not sure if it applies to all hosting environments, but it may be fixed for...
I've been encountering a similar issue when upgrading to 12.2, and started stripping back all the changes, to the point where I'm basically running the same exact code in 12.2 and 11.3 and it works in 11.3 but gives a similar error to the original qu...
