Is there a way in Azure to compare data in one field?

[CBull]

Is there a way to compare a time stamp within on field/column for an individual ID? For example, if I have two records for an ID and the time stamps are within 5 min of each other....I just want to keep the latest. But, for example, if they were an hour apart I would keep both records.

[merca]

Windowing function can be what you need.

from pyspark.sql import functions as F
df.groupBy(F.window("event_time","5 minutes"))

[CBull]

So, is this done something like this?

SELECT  

    r.patientmedicalrecordnumber,

    r.callreceiveddatetime as date

FROM  table r

    LEFT OUTER JOIN table p

          ON r.pageid = p.pageid

WHERE p.pagetype = 6

   and cast(r.callreceiveddatetime as date) = current_date() - 1

df.groupBy (r.window("event_time","5 minutes"))

ORDER BY r.callreceiveddatetime

[merca]

Since you are trying to do this in SQL, I hope someone else can write you the correct answer. The above example is for pyspark. You can check the SQL synax from Databricks documents

[Kaniz]

Hi @Cory Bullard​, We haven't heard from you on the last response from @Merca Ovnerud​, and I was checking back to see if you have a resolution yet.

If you have any solution, please share it with the community as it can be helpful to others. Otherwise, we will respond with more details and try to help.

Also, please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.