In my situation we cannot split this notebook as ADF pipeline is already in PROD ,I have tried to use the option %%capture .It helps to ran the notebook within size limits but somehow it is corrupting the output. Also checked in the Databricks AI and got this result ."The %capture magic command is used in Jupyter notebooks to capture the output of a cell and prevent it from being displayed. However, in Databricks notebooks, this magic command is not supported."