<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Populating a array of date tuples Scala in Data Engineering</title>
    <link>https://community.databricks.com/t5/data-engineering/populating-a-array-of-date-tuples-scala/m-p/16989#M11070</link>
    <description>&lt;P&gt;&lt;/P&gt;
&lt;P&gt;Hi Friends&lt;/P&gt;
&lt;P&gt;i am trying to pass a list of date ranges needs to be in the below format.&lt;/P&gt;
&lt;P&gt;val predicates =&lt;/P&gt;&lt;P&gt;&lt;/P&gt;Array(&lt;P&gt;&lt;/P&gt;“2021-05-16” → “2021-05-17”,&lt;P&gt;&lt;/P&gt;“2021-05-18” → “2021-05-19”,&lt;P&gt;&lt;/P&gt;“2021-05-20” → “2021-05-21”)
&lt;P&gt;I am then using map to create a range of conditions that will be passed to the jdbc method.&lt;/P&gt;
&lt;P&gt;val predicates =&lt;/P&gt;&lt;P&gt;&lt;/P&gt;Array(&lt;P&gt;&lt;/P&gt;“2021-05-16” → “2021-05-17”,&lt;P&gt;&lt;/P&gt;“2021-05-18” → “2021-05-19”,&lt;P&gt;&lt;/P&gt;“2021-05-20” → “2021-05-21”)…map { case (start, end) =&amp;gt;&lt;P&gt;&lt;/P&gt;s"cast(NEW_DT as date) &amp;gt;= date ‘$start’ AND cast(NEW_DT as date) &amp;lt;= date ‘$end’"
&lt;P&gt;The process will need to run daily and i need to dynamically populate these values as i cannot use the hard coded way.&lt;/P&gt;
&lt;P&gt;Need help in how i can return these values from a method with incrementing start_date and end_date tuples that can generate like above.I had a mere idea like below but as i am new to scala not able to figure out. Please help&lt;/P&gt;
&lt;P&gt;def predicateRange( start_date: String,&lt;/P&gt;&lt;P&gt;&lt;/P&gt;end_date: String): Array[(String,String)]= {&lt;P&gt;&lt;/P&gt;// iterate over the date values and add + 1 to both start and end and return the tuple&lt;P&gt;&lt;/P&gt;} 
&lt;P&gt;&lt;/P&gt;</description>
    <pubDate>Sat, 31 Jul 2021 06:32:26 GMT</pubDate>
    <dc:creator>Techmate</dc:creator>
    <dc:date>2021-07-31T06:32:26Z</dc:date>
    <item>
      <title>Populating a array of date tuples Scala</title>
      <link>https://community.databricks.com/t5/data-engineering/populating-a-array-of-date-tuples-scala/m-p/16989#M11070</link>
      <description>&lt;P&gt;&lt;/P&gt;
&lt;P&gt;Hi Friends&lt;/P&gt;
&lt;P&gt;i am trying to pass a list of date ranges needs to be in the below format.&lt;/P&gt;
&lt;P&gt;val predicates =&lt;/P&gt;&lt;P&gt;&lt;/P&gt;Array(&lt;P&gt;&lt;/P&gt;“2021-05-16” → “2021-05-17”,&lt;P&gt;&lt;/P&gt;“2021-05-18” → “2021-05-19”,&lt;P&gt;&lt;/P&gt;“2021-05-20” → “2021-05-21”)
&lt;P&gt;I am then using map to create a range of conditions that will be passed to the jdbc method.&lt;/P&gt;
&lt;P&gt;val predicates =&lt;/P&gt;&lt;P&gt;&lt;/P&gt;Array(&lt;P&gt;&lt;/P&gt;“2021-05-16” → “2021-05-17”,&lt;P&gt;&lt;/P&gt;“2021-05-18” → “2021-05-19”,&lt;P&gt;&lt;/P&gt;“2021-05-20” → “2021-05-21”)…map { case (start, end) =&amp;gt;&lt;P&gt;&lt;/P&gt;s"cast(NEW_DT as date) &amp;gt;= date ‘$start’ AND cast(NEW_DT as date) &amp;lt;= date ‘$end’"
&lt;P&gt;The process will need to run daily and i need to dynamically populate these values as i cannot use the hard coded way.&lt;/P&gt;
&lt;P&gt;Need help in how i can return these values from a method with incrementing start_date and end_date tuples that can generate like above.I had a mere idea like below but as i am new to scala not able to figure out. Please help&lt;/P&gt;
&lt;P&gt;def predicateRange( start_date: String,&lt;/P&gt;&lt;P&gt;&lt;/P&gt;end_date: String): Array[(String,String)]= {&lt;P&gt;&lt;/P&gt;// iterate over the date values and add + 1 to both start and end and return the tuple&lt;P&gt;&lt;/P&gt;} 
&lt;P&gt;&lt;/P&gt;</description>
      <pubDate>Sat, 31 Jul 2021 06:32:26 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/populating-a-array-of-date-tuples-scala/m-p/16989#M11070</guid>
      <dc:creator>Techmate</dc:creator>
      <dc:date>2021-07-31T06:32:26Z</dc:date>
    </item>
    <item>
      <title>Re: Populating a array of date tuples Scala</title>
      <link>https://community.databricks.com/t5/data-engineering/populating-a-array-of-date-tuples-scala/m-p/16990#M11071</link>
      <description>&lt;P&gt;So basically this can be done by generating 2 lists which are then zipped.&lt;/P&gt;&lt;P&gt;One list contains the first dates of the tuples, so these are in your case 2 days apart.&lt;/P&gt;&lt;P&gt;The other list is the 2nd dates of the tuples, also 2 days apart.&lt;/P&gt;&lt;P&gt;&lt;/P&gt;&lt;P&gt;Now we need a function which creates these lists.&lt;/P&gt;&lt;P&gt;Mind that I found this function on the internet somewhere (forgot where) but it is elegant and works great! &lt;/P&gt;&lt;P&gt;&lt;/P&gt;&lt;P&gt;import java.time.LocalDate&lt;/P&gt;&lt;P&gt;&lt;/P&gt;&lt;P&gt;//create an infinite stream of dates starting at `fromDate`.&lt;/P&gt;&lt;P&gt;//streams are lazy evaluated so they are only filled as requested&lt;/P&gt;&lt;P&gt;def dates(fromDate: LocalDate): Stream[LocalDate] = {&lt;/P&gt;&lt;P&gt;&amp;nbsp;fromDate #:: dates(fromDate plusDays 2 )  //add 2 days&lt;/P&gt;&lt;P&gt;}&lt;/P&gt;&lt;P&gt;&lt;/P&gt;&lt;P&gt;val x = dates(LocalDate.now()).take(5).toList //start from today&lt;/P&gt;&lt;P&gt;val y = dates(LocalDate.now().plusDays(1)).take(5).toList //start from tomorrow&lt;/P&gt;&lt;P&gt;&lt;/P&gt;&lt;P&gt;val list = x zip y&lt;/P&gt;</description>
      <pubDate>Fri, 10 Sep 2021 15:05:35 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/populating-a-array-of-date-tuples-scala/m-p/16990#M11071</guid>
      <dc:creator>-werners-</dc:creator>
      <dc:date>2021-09-10T15:05:35Z</dc:date>
    </item>
  </channel>
</rss>

