I have a time-series that I am trying to model using auto.arima in R. Upon execution on a cluster runtime with 96GB memory and 48 cores, I observed that the code kept running without any end in sight. It ran for over a day without finishing.
Curious, I ran the same code on the same time-series on my modest laptop (32GB RAM, 10 cores), the code finished execution in under 5 minutes.
I am at a complete loss to understand what is taking the code I am running on my Databricks cluster runtime take so long!
For completeness, the code I am trying to execute is simply this:
baselines_daily <- auto.arima(daily_network_efficiency,
xreg = F,
seasonal = FALSE,
stepwise = FALSE,
parallel = TRUE,
approximation = FALSE)
where daily_network_efficiency is my time-series.
Any help in shedding light on this mystery will be greatly appreciated.
Thank you!