- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-12-2022 05:26 AM
Hi @Borislav Blagoev ,
Thanks very much for taking the time to collect these logs.
The problem here (as indicated by the `IpAclValidation` message) is that IP allow listing (enabled for your workspace) will not allow arbitrary connections from Spark clusters back to the control plane by default. The subnet(s) for the data plane needs to be added to the IP allow list.
Here is a link to docs on IpACL validation: https://docs.databricks.com/security/network/ip-access-list.html
Here is a link to docs on managing subnets in the data plane: https://docs.databricks.com/administration-guide/cloud-configurations/aws/customer-managed-vpc.html#...
Not sure what your end goal is with this, but it's probably also worth mentioning that there are (better) alternatives to using the `databricks-sql-connector` on Databricks notebooks. For example, in a Python notebook you can just use `spark.sql(...)` to execute SQL commands.