Unittesting databricks.sdk.runtime

[samarth_solanki]

How to mock a code that uses dbutils from
"from databricks.sdk.runtime import dbutils"
it shows databricks-sdk has no attribute runtime

[Kaniz_Fatma]

Hi @samarth_solanki, 

When you’re dealing with code that uses dbutils from the Databricks SDK, it can be tricky to mock it for unit testing. Let’s explore a couple of approaches to address this issue:

1. Inject dbutils as a Parameter:

   • Instead of using dbutils globally, try to receive it as a parameter in your functions. This way, your code becomes more testable.
   • Modify your functions to accept dbutils as an argument. For example:

     def my_function(dbutils):
         # Your code that uses dbutils goes here
         dbutils.fs.ls("/tmp")
     

   • In your unit tests, pass a mock object for dbutils when calling your function.

2. Use a Mock Library:

   • The unittest.mock library provides a straightforward way to create mock objects.
   • Here’s an example of how you can use it:

     from unittest.mock import MagicMock
     from my_library import my_function
     
     def test_my_function_calls_dbutils():
         # Create a mock for dbutils
         mock_dbutils = MagicMock()
     
         # Call your function with the mock
         my_function(dbutils=mock_dbutils)
     
         # Assert that the expected method was called
         mock_dbutils.fs.ls.assert_called_once_with("/tmp")
     

3. Additional Resources:

   • If you’re working with Databricks notebooks locally, consider using a mock like DbutilsMock¹². This allows you to simulate dbutils behavior during testing.
   • You can find an example of using DbutilsMock in the GitHub repository³.

Remember that these approaches help you isolate your code for testing without relying on the actual dbutils implementation. Happy testing! 🧪🚀

[Ayushi_Suthar · Accepted solution]

Hi @samarth_solanki , I Hope you are doing well! 

Based on the information you have shared, it seems like you're trying to import dbutils from databricks.sdk.runtime, but you're encountering an error that says "databricks-sdk has no attribute runtime". 

This issue might be due to the incorrect import statement or the version of the Databricks SDK you're using.  To use dbutils with the Databricks SDK for Python is to call it from the WorkspaceClient. Kindly refer to this as a example: 

from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
d = w.dbutils.fs.ls('/')
for f in d:
    print(f.path)

If you still face the issue then I would suggest checking the latest version of the Databricks SDK you are using and you can upgrade to the latest version using the below command: 

%pip install databricks-sdk --upgrade

Please refer this document for the more details: https://docs.databricks.com/ja/dev-tools/sdk-python.html#use-databricks-utilities

Please let me know if this helps and leave a like if this information is useful, followups are appreciated.
Kudos
Ayushi