topic Re: Creating a python package that uses dbutils.secrets in Get Started Discussions

Creating a python package that uses dbutils.secrets

samarth_solanki — Wed, 14 Feb 2024 10:45:45 GMT

Hello Databricks,
I wanted to create python package which has a python script which has a class , this class is where we give scope and key and we het the secret. This package will be used inside a databricks notebook
I want to use dbutils.secret for this
Q1. I have used dbutils in my code , but when i import this package in notebook and run it it shows dbutils not defined, is it not possible it takes dbutils on itself when running this package in notebook.
Q2. Then I also added "from databricks.sdk.runtime import dbutils" but this throws error saying "cannot configure default auth"
I want to create this python package locally and run it in databricks notebook where it can use dbutils , what can i do?

Re: Creating a python package that uses dbutils.secrets

BR_DatabricksAI — Wed, 14 Feb 2024 12:10:27 GMT

Hello samarth_solanki,

Any specific reason for using the scope & key in class, however in data bricks and as a best practice we can create the two types of Databricks Secret Scopes:

Azure Key Vault-Backed Scope :

Users can create a Secret Scope backed by the Azure Key Vault. It allows users to leverage all the Secrets in the corresponding Key Vault instance from a particular Secret Scope.

Databricks-Backed Scope

In this method, the Secret Scopes are managed with an internally encrypted database owned by the Databricks platform. Users can create a Databricks-backed Secret Scope using the Databricks CLI .

This option would be viable for you in case if don't want to leverage the azure key vault option.

Thanks

Re: Creating a python package that uses dbutils.secrets

mfall-shift — Fri, 29 Mar 2024 16:25:33 GMT

@samarth_solanki wrote:
Hello Databricks,
I wanted to create python package which has a python script which has a class , this class is where we give scope and key and we het the secret. This package will be used inside a databricks notebook
I want to use dbutils.secret for this
Q1. I have used dbutils in my code , but when i import this package in notebook and run it it shows dbutils not defined, is it not possible it takes dbutils on itself when running this package in notebook.
Q2. Then I also added "from databricks.sdk.runtime import dbutils" but this throws error saying "cannot configure default auth"
I want to create this python package locally and run it in databricks notebook where it can use dbutils , what can i do?

Hello, I have exactly the same issue (in my case I want to use dbutils.fs.ls). Did you find a solution?

Re: Creating a python package that uses dbutils.secrets

heiko_u — Thu, 15 Aug 2024 21:12:15 GMT

Same question. Did you ever get an answer for this?

I want to use databricks secrets inside my custom Python code. This is only necessary since based on this post (https://community.databricks.com/t5/administration-architecture/aws-secrets-manager-access/td-p/50197), DBR > 12.2 LTS no longer allows the use of AWS Secret Manager

Re: Creating a python package that uses dbutils.secrets

krishnab — Tue, 10 Dec 2024 15:13:41 GMT

from pyspark.dbutils import DBUtils from pyspark.sql import SparkSession spark = SparkSession.builder.getOrCreate() dbutils = DBUtils(spark)

Adding this to the module file solves the problem.