Hi Yuki,

The primary reason for ML runtime not being supported on Shared mode cluster is security and resource isolation: 1) ML workloads frequently require privileged operations (e.g., running arbitrary code, installing dependencies) not compatible with the multi-user process isolation model of Standard mode.
2) Many ML libraries (especially GPU-enabled or native code) need access to the underlying filesystem or privileged resources, which could break the isolation guarantees required for data security/governance in Standard mode.
3) Supporting ML Runtime in Standard would open up nontrivial risk of privilege escalation or governance circumvention, and enforcement is difficult.

Whereas,

Dedicated mode provides each compute resource to a single principal (user or group). This mode:
1) Allows installation of arbitrary libraries, use of MLflow, custom environments, GPU acceleration, and access to features like DBFS/FUSE that multi-user safety would otherwise restrict.
2) Makes ML workloads possible while still integrating with Unity Catalog for data governance—albeit at the cost of simplified sharing and more manual folder/secret management.

I hope this clarifies your question!