I would guess that at this point, you will have to destroy them manually. In the future, you might want to let your TF script create all the requisite network resources; then the script will also be able to destroy them. This is a general GKE issue and not specific to Bricks. GKE does a lot of funny things at the network level and if you don't do them first, ha. 

If you look at the example in the provider here: https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/gcp-workspace

I think they have an example creating the network.