Cluster termination issue (It was working fine with same code previously)

Humi1245
New Contributor

‎09-26-2023 12:48 AM

Hi, I am running arima code on databricks commmunity edition. Last time, it worked but now the cluster gets terminated at this command and says its completed. but i cant further plot the predictions since the cluster is terminated. 

 

# Fit the AutoARIMA model
model = auto_arima(train['Price'], seasonal=True, m=365, stepwise=True, suppress_warnings=True, error_action='ignore', trace=False)
 
What is the issue?
0 Kudos

Kumaran
Databricks Employee
Databricks Employee

‎10-03-2023 07:46 AM - edited ‎10-03-2023 07:47 AM

Hi  @Humi1245 

Thank you for your question in the Databricks community

One possible issue may be that the cluster is running out of memory while trying to fit the AutoARIMA model. You can try increasing the size of the cluster and see if that resolves the issue.

You can check the status of the cluster, the resource utilization in the cluster and the network connectivity to the cluster.

Annapurna_Hiriy
Databricks Employee
Databricks Employee

‎10-11-2023 12:33 AM

@Humi1245 You should check the cluster logs to see the reason for cluster termination. Also, check the metrics to understand the memory/CPU consumption.

0 Kudos

Tharun-Kumar
Databricks Employee
Databricks Employee

‎10-11-2023 02:56 AM

@Humi1245 

You can view the event log of the cluster to understand the cause of the cluster termination. If it is due to gc issues, then we have to understand the reason behind high driver memory utilization which caused the GC. 

If it's due to inactivity, then the cluster has been idle for long time.

0 Kudos