Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
At my company we use conda-pack to make certain packages available on the spark executors. Is there a better a better alternative to get away from creating a new environment and pack it every time I need a new python lib to be available for the executors?
