I recently faced an issue that took good hours to identify.
I'm loading an environment variable with a secret
ENVVAR: {{secrets/scope/key}}
The secret is loaded in my application, I could verify it's there, but its value is not correct. I realised that after comparing the environment variable value with the value of the secret when loading using dbutils
envvar = dbutils.secrets.get(scope="scope", key="key")
After a lot of trial and error, I realised that the problem is because my secret value has a dollar sign in it, and when loading through env vars the dollar sign and everything after it disappears, I suspect that it's somehow evaluated as an environment variable and gets replaced as well.
Example:
print(dbutils.secrets.get(scope="scope", key="key"))
# this would be "mysecret$here"
print(os.environ["ENVVAR"])
# this would be "mysecret"
Is it a bug on Databricks or is it something else?
