Anonymous
Not applicable

@Sulfikkar Basheer Shylaja​ :

Yes, you can use the ClusterInitScript on a Databricks cluster that does not use a Docker image. The ClusterInitScript allows you to run a script on the nodes of your cluster when it starts up, which can be used to configure your cluster.

To use ClusterInitScript, you can follow these steps:

  1. Create a script that you want to run on each node of your cluster during initialization. This script can be a shell script or a Python script.
  2. Upload the script to a location that is accessible by all nodes of the cluster. You can upload the script to DBFS (Databricks File System), or to a cloud storage location such as Amazon S3 or Azure Blob Storage.
  3. Open your Databricks workspace and navigate to the cluster that you want to configure.
  4. Click on the "Advanced Options" tab.
  5. Scroll down to the "Init Scripts" section.
  6. Click "Add" to add a new initialization script.
  7. Enter the URL or path to the script that you uploaded in step 2.
  8. Optionally, you can specify arguments to pass to the script using the "Parameters" field.
  9. Click "Confirm" to save the initialization script.

When you start or restart your cluster, the initialization script will be run on each node of the cluster. You can check the logs of each node to ensure that the script was executed successfully.

Note that the ClusterInitScript feature is only available on Databricks clusters that are created using the VPC network configuration. If your cluster is not configured to use VPC, you will need to create a custom image with your initialization script baked in and use it to launch your cluster.