Private Route53 hosted zone for cloud.databricks.com

[margarita_shir]

We have Tailscale deployed on the same VPC as our Databricks workspaces. To enable PrivateLink for some workspaces, we created a private Route53 hosted zone for cloud.databricks.com with CNAME records pointing workspace hostnames to the PrivateLink endpoint.

The problem is that this private zone shadows all of *.cloud.databricks.com within the VPC, including:

• accounts.cloud.databricks.com — needed for public authentication
• ui-assets.cloud.databricks.com — CDN for UI static assets

For accounts.cloud.databricks.com we worked around this by adding an explicit A alias record in the private zone pointing to the public ELB. However, Databricks recently rotated that ELB hostname, breaking our hardcoded record.

For ui-assets.cloud.databricks.com — we only discovered today that this subdomain is also under cloud.databricks.com and was returning NXDOMAIN from the private zone, causing a blank page on all workspaces for Tailscale users.

Question: What is the recommended approach for maintaining a private hosted zone for cloud.databricks.com while keeping public subdomains like accounts and ui-assets resolvable? Is there a supported way to handle this without hardcoding ELB hostnames that can rotate?

Thank you,

[suranga]

The safer approach is not to shadow cloud.databricks.com with one private hosted zone. Databricks recommends the privatelink.cloud.databricks.com DNS pattern with conditional forwarding, which avoids hardcoding rotating ELB hostnames and keeps public services like accounts and ui-assets resolvable through normal public DNS.

[stbjelcevic]

Your current setup is a known anti-pattern. By creating a PHZ for cloud.databricks.com, you're shadowing every subdomain under that zone, including accounts, ui-assets, and any future public subdomains Databricks adds.

How it works

Databricks workspace URLs resolve through a CNAME chain:

<workspace>.cloud.databricks.com → <region>.privatelink.cloud.databricks.com

So the PHZ only needs to cover privatelink.cloud.databricks.com with regional A records pointing to your VPC endpoint private IPs. This leaves all public subdomains (accounts, ui-assets, etc.) resolving normally through public DNS.

Recommended Steps

1. Create a new PHZ for privatelink.cloud.databricks.com associated with your VPC
2. Add A records for your region(s), e.g. us-east-1.privatelink.cloud.databricks.com → VPC endpoint private IPs
3. Delete the old cloud.databricks.com PHZ
4. Verify accounts.cloud.databricks.com and ui-assets.cloud.databricks.com resolve publicly again

This eliminates all hardcoded ELB hostnames and is resilient to Databricks infrastructure rotations.

References
- https://docs.databricks.com/aws/en/security/network/classic/privatelink-dns
- https://community.databricks.com/t5/community-articles/configuring-dns-resolution-for-private-databr...

[margarita_shir]

Thank you for your response. I added a new private hosted zone in Amazon Route 53 for privatelink.cloud.databricks.com, and created an A record for us-east-1.privatelink.cloud.databricks.com pointing to the private endpoint IP.

I had a question regarding split DNS in Tailscale. It is currently configured to forward cloud.databricks.com to 10.104.248.2 (our VPC DNS resolver), as it was for the original private hosted zone. However, when I switch the split DNS domain to privatelink.cloud.databricks.com, it breaks private DNS resolution for workspaces using Private Link and starts returning public IPs instead. Does this mean we should keep the split DNS configuration in Tailscale set to cloud.databricks.com → 10.104.248.2?

[margarita_shir]

Thank you for your response. I added a new private hosted zone in Amazon Route 53 for privatelink.cloud.databricks.com, and created an A record for us-east-1.privatelink.cloud.databricks.com pointing to the private endpoint IP.

I had a question regarding split DNS in Tailscale. It is currently configured to forward cloud.databricks.com to 10.104.248.2 (our VPC DNS resolver), as it was for the original private hosted zone. However, when I switch the split DNS domain to privatelink.cloud.databricks.com, it breaks private DNS resolution for workspaces using Private Link and starts returning public IPs instead. Does this mean we should keep the split DNS configuration in Tailscale set to cloud.databricks.com → 10.104.248.2?

[suranga]

Hi @margarita_shir,

I don't think this follow-up got an answer, so jumping back in. Yes, keep the Tailscale split DNS on cloud.databricks.com pointing at your VPC resolver.

Here's why switching it to privatelink.cloud.databricks.com breaks things. Tailscale's split DNS only sends a query to your VPC resolver when the name being looked up matches the split domain. Your clients look up <workspace>.cloud.databricks.com, not the privatelink name. That query goes to your normal resolver, which follows the CNAME to <region>.privatelink.cloud.databricks.com on its own and returns the public answer, so the privatelink rule never kicks in. A corporate DNS server with a conditional forwarder behaves differently because it chases the CNAME itself, which is why the Databricks DNS docs list *.privatelink.cloud.databricks.com as one option. The same page also lists *.cloud.databricks.com, or one entry per workspace URL, as alternatives.

Forwarding cloud.databricks.com is safe now that your private zone only covers privatelink.cloud.databricks.com. The VPC resolver answers accounts and ui-assets from public DNS, and only the privatelink names get the private IP.

Two things worth checking:

Make sure the old cloud.databricks.com private hosted zone is deleted, or at least disassociated from the VPC. While it's still attached, it keeps shadowing accounts and ui-assets.
Make sure your A record matches the exact CNAME target. From outside the VPC, run nslookup on a workspace URL and look at the canonical name. The examples in the Databricks docs use a city name (sydney.privatelink.cloud.databricks.com), so yours may not be us-east-1.privatelink.cloud.databricks.com. If it doesn't match, the private zone shadows that name the same way the old zone shadowed accounts, and it won't resolve.
If you'd rather not send everything under cloud.databricks.com through the VPC resolver, add one split DNS entry per workspace URL instead.

Cheers,
Suranga