PGP decryption in python file

TetianaDromova — Wed, 09 Jul 2025 16:50:50 GMT

The same decryption code works in notebook, but fails in python file:

import gnupg
from pyspark.dbutils import DBUtils

dbutils = DBUtils(spark)

gpg = gnupg.GPG()

decryption_key = dbutils.secrets.get(secret_scope, secret_name)
gpg.import_keys(decryption_key)

input_file_path = f"/Volumes/{catalog}/{schema}/{volume}/{folder}/{file}.csv.pgp"

with open(input_file_path, 'rb') as f:
encrypted_data = f.read()

decrypted_data = gpg.decrypt(encrypted_data)

if decrypted_data.ok:
print("Ok")

So, in notebook it's Ok. But In python file I receive WARNING:gnupg:FAILURE status emitted from gpg process: decrypt 4294967295. And the decrypted_data is empty.

Re: PGP decryption in python file

mmayorga — Wed, 24 Sep 2025 00:17:36 GMT

Hi @TetianaDromova

Thank you for reaching out and waiting for a response.

Having your code working on a notebook is a significant first step, so you are on the right path, but then moving into a Python file, we must consider specific details:

How is this Python code/file being executed? In a Lakeflow Job as a task?
If it is in a Job, what user/service principal is configured to execute this job and task of your Python Code?
1. When you run code in a notebook, it uses your own credentials, which likely have all the required permissions. However, when running as a Job, the code may run under a different user or principal. You must ensure that this Job principal has the necessary permissions to access your volume in order to read the encrypted file. The same requirement applies to the access to the scope and secrets.
Are you using the same cluster with the same libraries installed to execute your notebook and Python files for consistency?

Based on the error provided, the issue likely occurs when accessing the volume or the secrets, which prevents proper decryption.

I hope this helps!

topic PGP decryption in python file in Data Engineering

PGP decryption in python file

Re: PGP decryption in python file