Databricks on AWS Marketplace – Unity Catalog & S3 Access Failing with SSL “Connection reset”
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
yesterday
Hi All,
I’m facing an issue accessing AWS S3 and Unity Catalog from a Databricks AWS Marketplace workspace.
I’m facing an issue accessing AWS S3 and Unity Catalog from a Databricks AWS Marketplace workspace.
Problem:
Whenever Databricks tries to access S3 or Unity Catalog, it fails with:
javax.net.ssl.SSLException: Connection reset
What works:
Spark jobs run fine
DNS resolution works
curl https://s3.us-east-1.amazonaws.com works from the cluster
NAT Gateway is configured
Outbound HTTPS (443) is allowed
What fails:
SHOW CATALOGS
dbutils.fs.ls("s3://...")
Creating schemas or tables in Unity Catalog
Environment:
AWS Marketplace Databricks (QuickLaunch)
Classic compute (Spark Connect)
Private subnets + NAT Gateway
Found an S3 Gateway VPC endpoint, removed its route table association and restarted cluster — issue still happens
Since basic network access works but Databricks-initiated calls fail, this looks like a Databricks / Unity Catalog connectivity issue, not IAM or general networking.
Has anyone faced this before?
Is S3 Gateway VPC endpoint supported with Unity Catalog in Marketplace workspaces?
Any guidance would really help. Thanks!
