Currently, in our company we are using ADF+DATABRICKS for all batch integration. Using ADF first data is copied to ADLS gen 2 (from different sources like on prem servers, ftp solution file sharing, etc), then it is reformatted to csv and it is copied to delta lake where all transformation and merging of data happens. Data is loaded to salesforce using custom bulk api v2 connector. We plan to remove this additional step of first copying data to ADLS gen 2 and rather do everything end to end ETL on databricks . could you please provide architecture diagram or pros/cons for this upgrade? fee free to ask question.

since it is all banking data, security might be a concern.