Hi There,
I'm currently going through Module 4 of the Data Engineering Associate pathway, specifically lesson 4.1 - DLT UI Walkthrough. We are instructed to specify the Cluster Policy as 'DBAcademy DLT' when configuring the pipeline. However, this option isn't listed as a Cluster Policy in the Compute section of the configuration settings. The only option available is 'None'.
I believe I have the correct cluster policies enabled when creating a cluster and am unable to find a solution.
It's a duplicate post and a solution have been provided in other post , which ask to run the below notebook.
/Users/<YOUR USER NAME>/Data Engineering with Databricks - v3.1.4/Includes/Workspace-Setup
Upon running it fails with below error. Please help.
DatabricksApiException: DatabricksApiException(message='You are not authorized to create instance pools. Please contact your administrator.',
