<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic auto.arima in R keeps running without end in Machine Learning</title>
    <link>https://community.databricks.com/t5/machine-learning/auto-arima-in-r-keeps-running-without-end/m-p/173607#M4732</link>
    <description>&lt;P&gt;I have a time-series that I am trying to model using auto.arima in R. Upon execution on a cluster runtime with 96GB memory and 48 cores, I observed that the code kept running without any end in sight. It ran for over a day without finishing.&lt;/P&gt;&lt;P&gt;Curious, I ran the same code on the same time-series on my modest laptop (32GB RAM, 10 cores), the code finished execution in under 5 minutes.&lt;/P&gt;&lt;P&gt;I am at a complete loss to understand what is taking the code I am running on my Databricks cluster runtime take so long!&lt;/P&gt;&lt;P&gt;For completeness, the code I am trying to execute is simply this:&lt;/P&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;baselines_daily &amp;lt;- auto.arima(daily_network_efficiency,&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; xreg = F,&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; seasonal = FALSE,&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; stepwise = FALSE,&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; parallel = TRUE,&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; approximation = FALSE)&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;&amp;nbsp;&lt;/DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;where&amp;nbsp;daily_network_efficiency is my time-series.&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;Any help in shedding light on this mystery will be greatly appreciated.&lt;/DIV&gt;&lt;DIV&gt;&amp;nbsp;&lt;/DIV&gt;&lt;DIV&gt;Thank you!&lt;/DIV&gt;&lt;/DIV&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
    <pubDate>Fri, 16 Oct 2026 10:41:59 GMT</pubDate>
    <dc:creator>arkobose</dc:creator>
    <dc:date>2026-10-16T10:41:59Z</dc:date>
    <item>
      <title>auto.arima in R keeps running without end</title>
      <link>https://community.databricks.com/t5/machine-learning/auto-arima-in-r-keeps-running-without-end/m-p/173607#M4732</link>
      <description>&lt;P&gt;I have a time-series that I am trying to model using auto.arima in R. Upon execution on a cluster runtime with 96GB memory and 48 cores, I observed that the code kept running without any end in sight. It ran for over a day without finishing.&lt;/P&gt;&lt;P&gt;Curious, I ran the same code on the same time-series on my modest laptop (32GB RAM, 10 cores), the code finished execution in under 5 minutes.&lt;/P&gt;&lt;P&gt;I am at a complete loss to understand what is taking the code I am running on my Databricks cluster runtime take so long!&lt;/P&gt;&lt;P&gt;For completeness, the code I am trying to execute is simply this:&lt;/P&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;baselines_daily &amp;lt;- auto.arima(daily_network_efficiency,&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; xreg = F,&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; seasonal = FALSE,&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; stepwise = FALSE,&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; parallel = TRUE,&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;&amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; &amp;nbsp; approximation = FALSE)&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;&amp;nbsp;&lt;/DIV&gt;&lt;DIV&gt;&lt;SPAN&gt;where&amp;nbsp;daily_network_efficiency is my time-series.&lt;/SPAN&gt;&lt;/DIV&gt;&lt;DIV&gt;Any help in shedding light on this mystery will be greatly appreciated.&lt;/DIV&gt;&lt;DIV&gt;&amp;nbsp;&lt;/DIV&gt;&lt;DIV&gt;Thank you!&lt;/DIV&gt;&lt;/DIV&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Fri, 16 Oct 2026 10:41:59 GMT</pubDate>
      <guid>https://community.databricks.com/t5/machine-learning/auto-arima-in-r-keeps-running-without-end/m-p/173607#M4732</guid>
      <dc:creator>arkobose</dc:creator>
      <dc:date>2026-10-16T10:41:59Z</dc:date>
    </item>
    <item>
      <title>Re: auto.arima in R keeps running without end</title>
      <link>https://community.databricks.com/t5/machine-learning/auto-arima-in-r-keeps-running-without-end/m-p/173615#M4733</link>
      <description>&lt;P&gt;Okay, this is resolved. Windows doesn't allow forking, so parallel = TRUE was having no effect. But Databricks runtime runs on Linux, which enables forking. So keeping parallel = TRUE seems to be having the opposite effect than the anticipated one: due to process overheads, and the fact that parallel only works when stepwise = FALSE, thereby forcing the model to perform a brute-force search.&lt;/P&gt;&lt;P&gt;Tl;dr: I turned off parallel and the execution finished fairly quickly.&lt;/P&gt;</description>
      <pubDate>Fri, 16 Oct 2026 12:36:13 GMT</pubDate>
      <guid>https://community.databricks.com/t5/machine-learning/auto-arima-in-r-keeps-running-without-end/m-p/173615#M4733</guid>
      <dc:creator>arkobose</dc:creator>
      <dc:date>2026-10-16T12:36:13Z</dc:date>
    </item>
  </channel>
</rss>

