Databricks serverless run init scripts

[prajwalpoojary]

Hello,

I want to read from Snowflake, which is in private network, using Databricks serverless compute.  We use init script to whitelist IP address inside the job/ interactive cluster. Now i am not able to execute init script inside serverless.

How to do this in serverless?

Script:

#!/bin/bash

cat /etc/hosts

cat <<EOT >> /etc/hosts

<ip_address>  <host>.snowflakecomputing.com

<ip_address> <host>.snowflakecomputing.com

EOT

Thanks

[juan_maedo]

 Hi  @prajwalpoojary , 

More than run init scripts, you might know and check all this.

Databricks Serverless compute uses dynamically assigned IP ranges each time a resource is attached. These ranges vary depending on the region and cloud provider you are using.

You need to allow these IP ranges in your different VNet services, firewalls, and in your Snowflake VNet as well. You may also need to include the NCC rules specific to your workspace. For more information, refer to the official documentation:

Databricks Serverless firewall configuration:
https://docs.databricks.com/aws/en/security/network/serverless-network-security/serverless-firewall

Default Databricks IP ranges by cloud provider:

• AWS: https://docs.databricks.com/aws/en/resources/ip-domain-region

• Azure: https://learn.microsoft.com/es-es/azure/databricks/resources/ip-domain-region
  

• GCP: https://docs.databricks.com/gcp/en/resources/ip-domain-region

Best,