Wassim
Databricks Partner

As you are talking about different schemas ,perhaps schemaevolutionmode, infercolumntypes, or schemahints may help?? Check out this- 32min onward - https://youtu.be/8a38Fv9cpd8

Hope it helps, do let know how you solve it if you can.​