This video provides a step-bystep guide on how to manually #deploy a Databricks workspace within an #AWS VPC. The tutorial concludes with the successful deployment of the #workspace.
Takeaways:
-Manually deploying a Databricks workspace requires several steps.
-Admin privileges in AWS are preferred for WS deployment so as not to run into any blockers due to a lack of permissions.
-A cross-account IAM role with an inline policy will be created for the deployment of EC2 instances, needed for classic compute
-A VPC will be deployed with the minimum needed configurations to host the EC2 instances that support classic compute in a secure manner. The VPC will have outbound connectivity to the internet. If you want no internet access at all, you'll need to implement backend Private Link. So the compute plane can communicate with the control plane privately; if not, the classic compute cluster's spin-up will fail.
-An S3 bucket will be created to serve as the root storage of the workspace, an IAM role and two inline policies will also be created so the workspace can interact with the S3 doing read and write operations
-If it's your first time creating a workspace, the metastore button will be grayed out, and the metastore will be created automatically. If you already have a metastore in your region, you just need to choose it.
-The Private Access Setting object is only needed if you want your users to connect to your workspace privately through a VPN; if not, it's not needed. You can implement after the workspace has been deployed if needed.
#databricks
Presented by Julian Camacho Cardoza
