Thank you for confirming the information in the opening post as accurate. The workarounds are not acceptable so feel free to close this issue or you can leave it open until a more mature solution is released on the Databricks platform.