โ07-26-2021 06:55 PM
Is there a way to Decrypt / Encrypt Blob files in Databricks using Key stored in Key Vault.
What libraries need to be used?
Any code snippets? Links?
โ09-02-2021 06:38 AM
Hi @ SreedharVengala! My name is Kaniz, and I'm the technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers on the Forum have an answer to your questions first. Or else I will follow up shortly with a response.
โ09-14-2021 04:54 PM
Thanks Kaniz... we got this working now using gnupg within databricks notebook.
Regards
Sreedhar
โ09-17-2021 02:45 PM
That's awesome!
Enjoy!
โ10-06-2021 10:33 AM
Hi @Sreedhar Vengalaโ , we are trying the same can you share some sample code on how we can achieve the same, and did you pick the files from ADLS and placed back the encrypted files to ADLS folders
โ11-18-2021 10:07 AM
Hi @Sreedhar Vengalaโ , Would you like to share the sample code here? It shall help other users.
โ11-16-2021 01:10 PM
Hi Team, Could anyone please help me on how to decrypt PGP keys using Azure Keyvault in Azure Databricks notebook.
โ11-21-2021 05:54 PM
Hi @Shobhit Awasthiโ , You can use the below snippet of code which doesn't need you to access anything from any directory other than the files you plan to encrypt and the keys. In the code, you have a public key stored in the key vault as a secret named 'publicb64'. If you want to read the ASC version from somewhere you can just read it into KEY_PUB. Don't forget to install pgpy using pip install pgpy.
#Encrypting a file using public key
import pgpy
from pgpy.constants import PubKeyAlgorithm, KeyFlags, HashAlgorithm, SymmetricKeyAlgorithm, CompressionAlgorithm
from timeit import default_timer as timer
import base64
import io
KEY_PUB = base64.b64decode(publicb64).decode("ascii").lstrip()
#print(KEY_PUB)
pub_key = pgpy.PGPKey()
pub_key.parse(KEY_PUB)
pass
# -READ THE FILE FROM MOUNT POINT-----------------
with io.open('/dbfs/mnt/sample_data/california_housing_test.csv', "r",newline='') as csv_file:
input_data = csv_file.read() # The io and newline retains the CRLF
t0 = timer()
#PGP Encryption start
msg = pgpy.PGPMessage.new(input_data)
###### this returns a new PGPMessage that contains an encrypted form of the original message
encrypted_message = pub_key.encrypt(msg)
pgpstr = str(encrypted_message)
with open('/dbfs/mnt/sample_data/california_housing_test.csv.pgp', "w") as text_file:
text_file.write(pgpstr)
print("Encryption Complete :" + str(timer()-t0))
โ11-22-2021 06:04 AM
@Kaniz Fatmaโ
Could you please share the decryption script on how to decrypt PGP keys using Azure Keyvault in Azure Databricks notebook.
โ12-03-2021 03:15 AM
Hi @Shobhit Awasthiโ , This might help you.
โ12-03-2021 10:53 AM
@Kaniz Fatmaโ
We are looking to decrypt using Azure Databricks notebook...the above git hub link is using Azure functions and .net libraries.
Could you please help on the same ?
โ12-14-2021 07:50 AM
Hi @Shobhit Awasthiโ , Does this help?
https://ization.in/2021/10/18/how-to-decrypt-pgp-encrypted-files-in-databricks/
โ04-06-2022 07:40 PM
@Kaniz Fatmaโ
Is the key generated via OpenPGP (Private and Public) or Python?
โ04-07-2022 12:05 AM
Hi @Alwin Leeโ , It is a Private Key, created alongside with public key and passphrase at the time of encryption.
โ04-07-2022 12:10 AM
@Kaniz Fatmaโ
Thank you.
Where is the key pairs were generated ? is it Python in Databricks or OpenPGP that created the key pairs?
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.