Rahul2025
Databricks Partner

Thank you @Sundar Raman​ for your response. Yes, we referred the same link that you shared and we currently use the Cluster-scoped script (following option #4) as against Global (following option #3) because the script needs to be executed only for one cluster and not all clusters that're part of a workspace. We're not using the following options #1 and #2 as they've been deprecated.

  1. Legacy global (deprecated)
  2. Cluster-named (deprecated)
  3. Global (new)
  4. Cluster-scoped

This clsuter-scoped init script has been provided from the UI and uploaded/deployed to the DBFS root. It has been uploaded/deployed to the DBFS root as described in the shared link (Section - Cluster-scoped init script locations). I'll check if referring it from ADLS directly helps address the size limitatation.