Hi Debayan,

Thank you for the reply and the links.

I have configured the workspace and Azure infrastructure as described in the links. All the storage and clusters were working except for the sample datasets.

I did a little digging into the firewall logs and found the following logs:

HTTPS request from 10.1.2.4:58006 to sts.amazonaws.com:443. Action: Deny. No rule matched. Proceeding with default action
 
HTTPS request from 10.1.2.4:41590 to databricks-datasets-oregon.s3.amazonaws.com:443. Action: Deny. No rule matched. Proceeding with default action
 
HTTPS request from 10.1.2.5:59056 to databricks-datasets-oregon.s3.us-west-2.amazonaws.com:443. Action: Deny. No rule matched. Proceeding with default action

So it seems the Azure workspace calls to AWS to read the sample datasets (I wouldn't want to be the one paying your data egress bill!).

I added the following rule to the firewall and it works:

      "Sample Datasets" = {
        action = "Allow"
        target_fqdns = [
          "sts.amazonaws.com",
          "databricks-datasets-oregon.s3.amazonaws.com",
          "databricks-datasets-oregon.s3.us-west-2.amazonaws.com"
        ]
        protocol = {
          type = "Https"
          port = "443"
        }
      }

Do you know what the mapping for Azure region to AWS region is? Looking at the docs here (https://docs.databricks.com/resources/supported-regions.html#control-plane-nat-and-storage-bucket-addresses) it seems you host the datasets across multiple regions. Do I have to whitelist all of them or does the Australia East region always go to Oregon to get the datasets?

Thanks,

Alex