topic Cannot Reproduce Result scikit-learn random forest in Data Engineering https://community.databricks.com/t5/data-engineering/cannot-reproduce-result-scikit-learn-random-forest/m-p/27657#M19518 <P>I'm running some machine learning experiments in databricks. For random forest algorithm when i restart the cluster, each time the training output is changes even though random state is set. Anyone has any clue about this issue?</P><P></P><P>Note : I tried the same algorithm with same code in anacoda enviroment in my local machine, there is no different in the result even though I restart the machine.</P><P></P><P></P> Fri, 18 Feb 2022 10:37:31 GMT umair 2022-02-18T10:37:31Z Cannot Reproduce Result scikit-learn random forest https://community.databricks.com/t5/data-engineering/cannot-reproduce-result-scikit-learn-random-forest/m-p/27657#M19518 <P>I'm running some machine learning experiments in databricks. For random forest algorithm when i restart the cluster, each time the training output is changes even though random state is set. Anyone has any clue about this issue?</P><P></P><P>Note : I tried the same algorithm with same code in anacoda enviroment in my local machine, there is no different in the result even though I restart the machine.</P><P></P><P></P> Fri, 18 Feb 2022 10:37:31 GMT https://community.databricks.com/t5/data-engineering/cannot-reproduce-result-scikit-learn-random-forest/m-p/27657#M19518 umair 2022-02-18T10:37:31Z Re: Cannot Reproduce Result scikit-learn random forest https://community.databricks.com/t5/data-engineering/cannot-reproduce-result-scikit-learn-random-forest/m-p/27659#M19520 <P>RF is non-deterministic by it´s nature.</P><P>However as you mentioned you can control this by using random_state.</P><P>This will guarantee a deterministic result ON A CERTAIN SYSTEM, but not necessarily over systems.</P><P><A href="https://stackoverflow.com/questions/70020127/is-result-of-train-test-split-the-same-on-different-machines-with-set-random-sta" alt="https://stackoverflow.com/questions/70020127/is-result-of-train-test-split-the-same-on-different-machines-with-set-random-sta" target="_blank">SO has a topic about this, check it out, very interesting</A>.</P><P></P> Mon, 21 Feb 2022 10:43:33 GMT https://community.databricks.com/t5/data-engineering/cannot-reproduce-result-scikit-learn-random-forest/m-p/27659#M19520 -werners- 2022-02-21T10:43:33Z