topic Re: Having a different performance while use GPU and CPU in Machine Learning

Having a different performance while use GPU and CPU

dangkhai — Fri, 22 Aug 2025 03:30:05 GMT

I'm building a model that is mostly sklearn libraries, but I'm also using TF-IDF and RandomForest. In theory, they only need a CPU to work properly, but in fact, when I use a physical computer with about 32 GB of RAM, it runs very fast. There are some strange things that happen when:
- I run the notebook with a compute CPU - 16 GB RAM - 4 cores on databricks and the result returns about 0.15s. However, my colleague used a calculator with the same configuration and ran it for about > 40s. This is the first oddity, the context is the same source code, the same model, the same kind of compute but there's a difference when the model predicts as above.
- When I perform the above model serving with CPU 0-64 cons, the model seems to have insufficient resources to predict and does not respond back to me. I don't think my model would work with that much resources. When I switched to the GPU, it worked as expected. Checking GPU Usage or memory is absolutely zero. My model is not using GPUs but it only works well on GPUs.

Can anyone answer the above questions for me?

Re: Having a different performance while use GPU and CPU

BR_DatabricksAI — Fri, 22 Aug 2025 08:25:22 GMT

Hello @dangkhai : I don't have much insight on your configuration used in your model. It all depends on the configuration what you are using in your RF i.e hyperparameters, depths etc. As you clearly mention that you are getting good resulted in GPU but not in CPU.

Can you try running your model with CPU based cluster with minor tweak using parallelism and see if that work out?

Re: Having a different performance while use GPU and CPU

dangkhai — Fri, 22 Aug 2025 08:59:57 GMT

Hi Sir,

Regarding your question "what you are using in your RF i.e. hyperparameters, depths etc.", I would like to share the following points about my training process:

Using 30,000 features for the TF-IDF matrix
Setting n_estimators = 500

Since I am working on a serverless environment in Databricks, I am not quite sure what you meant by "CPU based cluster with minor tweak using parallelism". Could you please provide more details on this so I can better understand your point?

Thank you for your support.

Best regards,

Re: Having a different performance while use GPU and CPU

BR_DatabricksAI — Fri, 22 Aug 2025 09:17:58 GMT

Hello @dangkhai : Have the below link and see parallelism enabled during transformation.

ParallelTextProcessing/parallelizing_text_processing.ipynb at master · rafaelvalero/ParallelTextProcessing