cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
ksilva
by • New Contributor
  • 3478 Views
  • 4 replies
  • 1 kudos

Incorrect secret value when loaded as environment variable

I recently faced an issue that took good hours to identify. I'm loading an environment variable with a secretENVVAR: {{secrets/scope/key}}The secret is loaded in my application, I could verify it's there, but its value is not correct. I realised tha...

Data Engineering
Reply
  • 3478 Views
  • 4 replies
  • 1 kudos
View Replies
Latest Reply
danmlopsmaz
New Contributor II
  • 1 kudos

Hi team, is there an update or fix for this?

  • 1 kudos
3 More Replies
Anonymous
by • Not applicable
  • 2197 Views
  • 3 replies
  • 5 kudos

What's the recommended way to use secrets within Global init scripts?

Data Engineering
Reply
  • 2197 Views
  • 3 replies
  • 5 kudos
View Replies
Latest Reply
sstrong99
New Contributor II
  • 5 kudos

I also have this question, and I don't understand the answer from @Sajith Appukuttan​ . This solution would require configuring each new cluster separately with the correct environment variable, correct? That seems like it defeats the purpose of usi...

  • 5 kudos
2 More Replies
Soma
by • Valued Contributor
  • 3781 Views
  • 3 replies
  • 3 kudos

Resolved! Unable to create Key Vault secrets scope with NPIP Workspace

Hi Team for secure connection we created secured cluster withNPIP(https://learn.microsoft.com/en-us/azure/databricks/security/secure-cluster-connectivity) WORKSPACE hosted in a private VNET.We had a hub vnet with private endpoint for key vault ,We pe...

Data Engineering
Reply
  • 3781 Views
  • 3 replies
  • 3 kudos
View Replies
Latest Reply
Debayan
Databricks Employee
  • 3 kudos

Hi @somanath Sankaran​ , did you face any error? if yes, could you please paste the error snapshot here?

  • 3 kudos
2 More Replies
rgrosskopf
by • New Contributor II
  • 4791 Views
  • 2 replies
  • 1 kudos

How to access secrets in Hashicorp Vault from Databricks notebooks?

I see in this blog post that Databricks supports Hashicorp Vault for secrets storage but I've been unable to find any additional details on how that would work. Specifically, how would I authenticate to Vault from within a Databricks notebook?

Data Engineering
Reply
  • 4791 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Hi @Ryan Grosskopf​,Just a friendly follow-up. Did any Prabakar's responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 1 kudos
1 More Replies
fermin_vicente
by • New Contributor III
  • 5109 Views
  • 7 replies
  • 4 kudos

Resolved! Can secrets be retrieved only for the scope of an init script?

Hi there, if I set any secret in an env var to be used by a cluster-scoped init script, it remains available for the users attaching any notebook to the cluster and easily extracted with a print.There's some hint in the documentation about the secret...

Data Engineering
Reply
  • 5109 Views
  • 7 replies
  • 4 kudos
View Replies
Latest Reply
pavan_kumar
Contributor
  • 4 kudos

@Fermin Vicente​ good to know that this approach is working well. but please make sure that you use this approach at the end of your init script only

  • 4 kudos
6 More Replies
sandip_yadav
by • New Contributor
  • 718 Views
  • 0 replies
  • 0 kudos

Databricks secrets visible in cleartext

I have a requirement that I need a secret when starting a cluster in databricks. And I found following way of providing the secret to my init script. https://docs.databricks.com/security/secrets/secrets.html#store-the-path-to-a-secret-in-an-environm...

Data Engineering
Reply
  • 718 Views
  • 0 replies
  • 0 kudos
User16790091296
by • Contributor II
  • 2422 Views
  • 1 replies
  • 1 kudos

Secrets in databricks

I created a secret on databricks using the secrets API.Code :Scope_name : {"scope": "dbtest", "initial_manage_principal":"user"} Resp= requests.post('https://instancename.net/mynoteid/api/2.0/secrets/scopes/create',json=Scope_name)Similar way, I adde...

Data Engineering
Reply
  • 2422 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
aladda
Databricks Employee
  • 1 kudos

You'll have to specify the scope and the key in the format below to get the value. dbutils.secret.get(scope="dbtest", key="user") Probably a good idea to review the Secret Management documentation for details on how to get this setup the right way - ...

  • 1 kudos
SimonNuss
by • New Contributor II
  • 29316 Views
  • 6 replies
  • 5 kudos

Resolved! Databricks cannot access Azure Key Vault

I am trying to set retrieve a secret from Azure Key Vault as follows: sqlPassword = dbutils.secrets.get(scope = "Admin", key = "SqlPassword") The scope has been created correctly, but I receive the following error message: com.databricks.common.clie...

Data Engineering
Reply
  • 29316 Views
  • 6 replies
  • 5 kudos
View Replies
Latest Reply
virahkumar
New Contributor II
  • 5 kudos

Sometimes turning it off and on again is underrated, so I gave up finding the problem, deleted it and re-created the scope - worked a breeze!Mine seems like it was something silly, I was able to set up my vault but got the same issue when trying to ...

  • 5 kudos
5 More Replies
Labels