How to build architecture for Batch as well Stream Data Pipeline in Databricks

Pratikmsbsvm
Contributor

Hello,

I am planning to Create a Data Lake house using Azure and Databricks.

Earlier i planned to do with Azure, but use cases looks complex.

Can someone please help me with suggestions.

Source System : SAP, SALESFORCE, SAP CAR, Adobe Clickstream.

Consumer : Salesforce, Spryker, Mad Mobile [API led Integration]

How to Handle Analytical Data

How to Handle Transactional data

Error Handling and connectivity

Real time data consume by spryker every 15 sec.

Thanks a lot for suggestion