Thanks for the info and your response — everything is clear now.

In short, I wanted to know if it’s possible to run a compiled Rust binary in Databricks. I tested locally and was able to process large datasets without Spark. Pandas wasn’t enough, Polars in Python worked, but Rust with Polars gave much better performance.

I was hoping to use that binary in Databricks without a full cluster, as a single machine with Rust was enough. But from your explanation, I understand that’s not currently possible.

So, if I want to leverage Rust, I should create bindings with Python (e.g., using PyO3) and call it from scripts or notebooks, combining Rust’s performance with Python’s ease of use.