I need to increase the stack size (from the default of 16384) to run a subprocess that requires a larger stack size.

I tried following this: https://community.databricks.com/t5/data-engineering/increase-stack-size-databricks/td-p/71492

And this: https://rangareddy.github.io/SparkStackOverflow/

But that didn't work.

I don't see any other solutions online.

Any ideas what to do? This seems like it should be possible to do.