@prashant151 - Unlike legacy (pre-UC) clusters, you cannot directly run a Databricks notebook (like setup.py) from a cluster init script, because init scripts only support shell commands — not %run or notebook execution.

You will need to refactor your setup logic into a Python module and install it via the init script.

I would do below instead

• Refactor steup.py into python package (.whl)
• Store your .whl to UC Volume
• Install via Cluster Init Script (Attach this init script to your cluster under Advanced Options → Init Scripts)
• Now you can import it in your notebook instead of %run

You can’t “%run a notebook” from a cluster init script—init scripts are shell-only and meant for environment setup (install libs, set env vars), not for executing notebooks or sharing Python state across sessions. +1 to what @Raman_Unifeye has told.

Convert your common code into a Python module or wheel and import it in notebooks instead of %run.

Option 1: Workspace modules (no build step)

Move Common/01_Utilities.py, 02_Utilities.py into Workspace Files as .py modules (add init.py). Then import them directly (DR 11.3+ auto-adds CWD to PYTHONPATH).

Example:

from Common.utilities import init_env, foo
init_env()

Option 2: Build a wheel and install per cluster

Package your code (setup.py or pyproject.toml), build a wheel, then install it as a cluster library or via %pip install /Workspace/.../yourpkg.whl (or from a UC volume).
Example:

%pip install /Workspace/Common/dist/yourpkg-0.1.0-py3-none-any.whl
import yourpkg
yourpkg.init_env()