Cluster access restrictions are more on a user/group-based level. Meaning that as the creator of the job, you'll be able to select the default cluster to run on based no the cluster that are visible to you. If another user wants to manually run the job, he'll be able to select a cluster which only he has access to.

More info on cluster access policies here: https://docs.databricks.com/administration-guide/clusters/policies.html