@Marcin Sieradzan​ :

The "Security Daemon Registration Exception" error occurs when the Databricks Security Agent running on the VPC can't register itself with the Databricks Control Plane. This error can happen due to a variety of reasons, such as incorrect network configuration or firewall rules.

Here are some steps to troubleshoot and resolve this issue:

1. Ensure that the Databricks Security Agent is running on the instances within the VPC that you want to connect to Databricks. You can check this by running the following command on an instance within the VPC: sudo systemctl status databricks-security-agent
2. Make sure that your VPC is properly configured with the correct route tables and network access control lists (NACLs) to allow traffic to and from the Databricks Control Plane. Check that you have created and attached a PrivateLink endpoint to your VPC.
3. Verify that the DNS resolution is working properly. You can do this by running the following command on an instance within the VPC: nslookup databricks-control-plane-aws-global.cloud.databricks.com. This command should return the Private IP address of the PrivateLink endpoint you have created.
4. Ensure that you have properly configured the security group associated with the Databricks workspace to allow traffic from the VPC.
5. Check that there are no firewall rules or network security rules that are blocking traffic from the VPC to the Databricks Control Plane.
6. Verify that you have the correct IAM roles and permissions set up in your AWS account to allow the necessary actions for PrivateLink. Check that the VPC endpoint policy allows access from the IAM role.
7. Finally, try creating the cluster again and monitor the logs for any error messages.