@nsingh_tl these results support the regional-DNS explanation @data_pulse raised. Your logs show public DNS resolution, not a missing route-table entry.

Databricks documents UC initialization contacting a regional hostname directly, bypassing the workspace URL’s PrivateLink CNAME chain. If the failing UC request uses that hostname, this could explain the difference between your REST and SQL results.

Since you already reported private DNS enabled, I’d check the actual workspace REST API endpoint’s DNS configuration, rather than repeat the workspace lookup. Run this read-only AWS CLI command from an authorized AWS terminal, replacing the endpoint ID:

aws ec2 describe-vpc-endpoints \
  --region us-west-1 \
  --vpc-endpoint-ids "<workspace-rest-vpce-id>" \
  --query 'VpcEndpoints[0].{PrivateDNS:PrivateDnsEnabled,DNSNames:DnsEntries[].DnsName}' \
  --output json

If PrivateDNS is false, enable it on that workspace endpoint through VPC > Endpoints > Actions > Modify private DNS name. Both VPC DNS resolution and DNS hostnames must be enabled. AWS documents these requirements here.

If it is already true, check whether the private DNS configuration covers the hostname UC actually uses, and inspect Route 53 Resolver rules associated with the VPC. A forwarding rule for the same domain can take precedence over a private hosted zone, so an enabled endpoint setting alone does not establish which DNS answer the cluster receives. AWS explains that precedence here.