Hi @Debayan Mukherjee​ ,

Thank you for your response, I have read this article and here are my understanding, please feedback if they are correct:

1. By default, all Python codes are run on the set of dependencies pre-installed in the runtime. Of course they can be different from the versions that developers use outside of Databricks.
2. The magic commands %conda and %pip only install specific dependencies for the notebook in which these commands are run.
3. Based on #2 above, these commands will not make sure all the modules inside the project to run on the same set of dependencies. I cannot use the method mentioned in Figure 5, because it is only for notebooks and not for normal python scripts which also need libraries on their own.
4. "Currently, %conda activate and %conda env create are not supported." Hence there is no solution to create and use a single virtual environment throughout the project?

Can you help with some additional questions:

1. When will the items in #4 above be made available (this article is dated 3 years ago)? This will ensure that the runtime environment and dev environment are consistent and avoid compatibility issues + avoid manual installation of additional/different packages one by one.
2. On this statement, "If you need some libraries that are always available on the cluster, you can install them in an init script or using a docker container.", these solutions will install libraries into the cluster with no ability to separate into different environments for different projects using the same cluster? If this is the case, then everyone using the same cluster will have to use the last set of libraries that got installed?
3. Are their any tools on Azure DB for me to create and manage virtual environments, not installing packages to the system environment?

Thank you for your kind assistance.