https://community.databricks.com/t5/data-engineering/how-to-build-data-pipeline-to-consume-data-from-adobe-campaign/m-p/130317#M48753 <P>Hello&nbsp;<a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/179126">@Datalight</a>&nbsp;<BR /><BR />Good day!<BR /><BR />Can I please know what do you mean by "you dont have any control over adobe"?&nbsp;<BR /><BR />I found similar case study over here:&nbsp;<A href="https://learn.microsoft.com/en-us/answers/questions/5533633/data-pipeline-to-push-files-from-external-system-(" target="_blank">https://learn.microsoft.com/en-us/answers/questions/5533633/data-pipeline-to-push-files-from-external-system-(</A>&nbsp;</P> Mon, 01 Sep 2025 08:05:40 GMT Khaja_Zaffer 2025-09-01T08:05:40Z https://community.databricks.com/t5/data-engineering/how-to-build-data-pipeline-to-consume-data-from-adobe-campaign/m-p/130301#M48747 <P>May Techie please help me design the pipeline with Databricks.</P><P>I don't have any control over Adobe.</P><P><SPAN>How to set up a data pipeline that moves csv files from Adobe to ADLS Gen2 via a cron job, using Databricks.</SPAN></P><P><SPAN>where this cron job will execute ?&nbsp;how ADLS Gen2 can detect new files.</SPAN></P><P><SPAN>How is the data being transferred from Adobe to ADLS Gen2 using Databricks only.</SPAN></P><P><SPAN>How to handle New File arrival?&nbsp;</SPAN></P><P><SPAN>I am trying to have Event Driven Architecture.</SPAN></P><P><STRONG>Limitation : The Workspace is not have Unity Catalog Enabled.</STRONG></P><P><STRONG>All the resources running on private subnets. No Public IP- NPIP = Yes</STRONG></P> Mon, 01 Sep 2025 05:49:45 GMT https://community.databricks.com/t5/data-engineering/how-to-build-data-pipeline-to-consume-data-from-adobe-campaign/m-p/130301#M48747 Datalight 2025-09-01T05:49:45Z https://community.databricks.com/t5/data-engineering/how-to-build-data-pipeline-to-consume-data-from-adobe-campaign/m-p/130317#M48753 <P>Hello&nbsp;<a href="https://community.databricks.com/t5/user/viewprofilepage/user-id/179126">@Datalight</a>&nbsp;<BR /><BR />Good day!<BR /><BR />Can I please know what do you mean by "you dont have any control over adobe"?&nbsp;<BR /><BR />I found similar case study over here:&nbsp;<A href="https://learn.microsoft.com/en-us/answers/questions/5533633/data-pipeline-to-push-files-from-external-system-(" target="_blank">https://learn.microsoft.com/en-us/answers/questions/5533633/data-pipeline-to-push-files-from-external-system-(</A>&nbsp;</P> Mon, 01 Sep 2025 08:05:40 GMT https://community.databricks.com/t5/data-engineering/how-to-build-data-pipeline-to-consume-data-from-adobe-campaign/m-p/130317#M48753 Khaja_Zaffer 2025-09-01T08:05:40Z