ModuleNotFoundError: No module named 'pyspark.dbutils'

[vk217]

I have a class in a python file like this

from pyspark.sql import SparkSession
from pyspark.dbutils import DBUtils
 class DatabricksUtils:

      def __init__(self‌‌):
        self.spark = SparkSession.getActiveSession()
        self.dbutils = DBUtils(self.spark)
     
     def get_dbutils(self) -> DBUtils:
        return self.dbutils

In another python file, I am importing this module and calling the db utils like

 from .myProject.functions.utils import *

    db = DatabricksUtils()

    dbutils = db.get_dbutils()

This works when I test it locally in vscode but in Azure Pipeline when I try to build my unit tests fail and I get a message

ModuleNotFoundError: No module named 'pyspark.dbutils'

[Miguel_Suarez]

Hey @vk217 ,

What Databricks runtime version was the cluster you ran the code on? I was able to successfully run your code in a 12.2 and a 13.3 cluster.

Can you try running it on a cluster with one of those DBR versions, if you haven't already? And please let us know if you're still running into issues.

Best,

Miguel

[Poonam_Joshi]

We are trying to do similar thing . we are using dbutils to get secret scope and we doing unittesting in azure pipeline where it gives error pyspark.dbutils not found 

We have tried using databricks.sdk.dbutils for which I got authentication error li kr value not found I used databricks -connect library as well but still the same issue..

Can you please help with it?

[Jarkrung]

Hi, we are also in the same exact situation. Were you able to solve the problem? Or a workaround maybe.