OP didnt wnat to expose environment variables:

"I am seeking advice on how to use these secrets in the `profiles.yml` file without exposing them as environment variables within Databricks. I need to ensure that these sensitive credentials remain secure while running through the Databricks workflow."

Is there another solution. like accessing secret scopes?

Best regards