@yit I've just been introduced to the VARIANT datatype on the course I'm doing. @szymon_dybczak  is bang on the mark here. This seems perfect!
https://www.youtube.com/watch?v=fWdxF7nL3YI  
https://www.databricks.com/blog/introducing-open-variant-data-type-delta-lake-and-apache-spark 

Really impressed with the speed benchmarking on the 2nd link! Can you let us know if this resolves the problem. Very interested 👌.

All the best,
BS