topic Re: Adobe query support from databricks in Data Engineering

Adobe query support from databricks

vickytscv — Tue, 10 Dec 2024 05:25:02 GMT

Hi Team,

We are working with Adobe tool for campaign metrics. which needs to pull data from AEP using explode option, when we pass query it is taking long time and performance is also very. Is there any better way to pull data from AEP, Please let let me know if there is best way to pull data from adobe tool.

#Adobe #query #ldatabricks

Re: Adobe query support from databricks

Alberto_Umana — Tue, 10 Dec 2024 17:55:36 GMT

Hi @vickytscv,

How are you integrating Adobe tool with Databricks, via a SQL warehouse? If so, could you please advise what is the statementID of the query with low performance, as well as the settings of the warehouse?

Re: Adobe query support from databricks

vickytscv — Wed, 11 Dec 2024 11:10:59 GMT

we are using below query for getting data from Adobe, here tis queries run in loop for every segment which is mentioned as parameter. Please help how we can get data in better way

SELECT

sm.value.lastQualificationTime,

sm.value.status,

sm.key,

ds.segment,

ds.segment_name,

sm.email,

sm.region

FROM (

SELECT

EXPLODE(value),

email,

region

FROM (

SELECT

EXPLODE(Segmentmembership),

_lifetech.UnifiedContactProfile.emailAddress AS email,

_lifetech.UnifiedContactProfile.contactRegion AS region

FROM

profile_snapshot_export_dbd09966_53c4_

WHERE

_lifetech.UnifiedContactProfile.emailAddress IS NOT NULL

)

) sm

JOIN adwh_dim_segments ds

ON sm.key = ds.segment

WHERE

sm.key LIKE '%{segmentid}%'

AND sm.value.lastQualificationTime > '{lastQualificationTime}'

) dbtable1

Re: Adobe query support from databricks

jodbx — Wed, 11 Dec 2024 13:54:56 GMT

https://github.com/Adobe-Marketing-Cloud/aep-cloud-ml-ecosystem