Cluster termination issue (It was working fine with same code previously)

[Humi1245]

Hi, I am running arima code on databricks commmunity edition. Last time, it worked but now the cluster gets terminated at this command and says its completed. but i cant further plot the predictions since the cluster is terminated. 

# Fit the AutoARIMA model

model = auto_arima(train['Price'], seasonal=True, m=365, stepwise=True, suppress_warnings=True, error_action='ignore', trace=False)

What is the issue?

[Kumaran]

Hi  @Humi1245 

Thank you for your question in the Databricks community

One possible issue may be that the cluster is running out of memory while trying to fit the AutoARIMA model. You can try increasing the size of the cluster and see if that resolves the issue.

You can check the status of the cluster, the resource utilization in the cluster and the network connectivity to the cluster.

[Annapurna_Hiriy]

@Humi1245 You should check the cluster logs to see the reason for cluster termination. Also, check the metrics to understand the memory/CPU consumption.

[Tharun-Kumar]

@Humi1245 

You can view the event log of the cluster to understand the cause of the cluster termination. If it is due to gc issues, then we have to understand the reason behind high driver memory utilization which caused the GC. 

If it's due to inactivity, then the cluster has been idle for long time.