I think adding this to my databricks.yml file helped. I probably did have a permissions problem and just granting these permissions over the whole of the bundle solved my issue.

permissions:
- group_name: my-admin-group
  level: CAN_MANAGE
- group_name: my-user-group
  level: CAN_VIEW