I'm building a model that is mostly sklearn libraries, but I'm also using TF-IDF and RandomForest. In theory, they only need a CPU to work properly, but in fact, when I use a physical computer with about 32 GB of RAM, it runs very fast. There are some strange things that happen when:
- I run the notebook with a compute CPU - 16 GB RAM - 4 cores on databricks and the result returns about 0.15s. However, my colleague used a calculator with the same configuration and ran it for about > 40s. This is the first oddity, the context is the same source code, the same model, the same kind of compute but there's a difference when the model predicts as above.
- When I perform the above model serving with CPU 0-64 cons, the model seems to have insufficient resources to predict and does not respond back to me. I don't think my model would work with that much resources. When I switched to the GPU, it worked as expected. Checking GPU Usage or memory is absolutely zero. My model is not using GPUs but it only works well on GPUs.
Can anyone answer the above questions for me?
