Re: Python file testing using pytest

sudhanshu1 · ‎09-19-2023

Hi All,

I have a requirement in my project, where we will be writing some python code inside databricks . Please note we will not be using pyspark . It will plain pythin with polars.

I am looking into ho to create test files for main file. Below is simple example which i am trying

Inside repo - i created a file called function_file.py. I wrote a simple function

def name_function(name):

return name + 'test'

Now i created another file called test_file.py and wrote below code

import pytest

from function_file import *

def test_name():

assert 1==1

Now i create a notebook and try to run below code

import pytest

import os

import sys

repo_name = "databricks_test_practice"

# Get the path to this notebook, for example "/Workspace/Repos/{username}/{repo-name}".

notebook_path = dbutils.notebook.entry_point.getDbutils().notebook().getContext().notebookPath().get()

# Get the repo's root directory name.

repo_root = os.path.dirname(os.path.dirname(notebook_path))

print(repo_root)

# # Prepare to run pytest from the repo.

os.chdir(f"/Workspace{repo_root}")

print(os.getcwd())

# # Skip writing pyc files on a readonly filesystem.

sys.dont_write_bytecode = True

# # Run pytest.

retcode = pytest.main([".", "-v", "-p", "no:cacheprovider"])

# # Fail the cell execution if there are any test failures.

assert retcode == 0, "The pytest invocation failed. See the log for details."

----------------------------

But i am getting below error .

file /Workspace/Repos/sudhanshu.raj@zuhlke.com/databricks_test_practice/tests/test_file.py, line 4 def test_name(): E fixture 'name' not found > available fixtures: capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory > use 'pytest --fixtures [testpath]' for help on them.

sudhanshu1 · ‎09-19-2023

Thanks Kaniz or your answer. However I think i am struggling to understand some basics.

Currently i just wrote a simple test function.

def test_something():

assert 1==1

This test passes , but when i make any small change in this function ,it doesnt reflect .

Example i change assert to 1==0, but my output is still same.

How to refresh main notebook which i am using to run tests files to pick latest change from tests files please? One solution is to restart cluster every time , but that's very bad solution

tomcorbin · ‎12-15-2023

Hi @sudhanshu1,

Try running dbutils.library.restartPython() in your notebook after you make any changes to the function_file.py file.