Databricks

sudhanshu1 · ‎09-19-2023

Hi All,

I have a requirement in my project, where we will be writing some python code inside databricks . Please note we will not be using pyspark . It will plain pythin with polars.

I am looking into ho to create test files for main file. Below is simple example which i am trying

Inside repo - i created a file called function_file.py. I wrote a simple function

def name_function(name):

return name + 'test'

Now i created another file called test_file.py and wrote below code

import pytest

from function_file import *

def test_name():

assert 1==1

Now i create a notebook and try to run below code

import pytest

import os

import sys

repo_name = "databricks_test_practice"

# Get the path to this notebook, for example "/Workspace/Repos/{username}/{repo-name}".

notebook_path = dbutils.notebook.entry_point.getDbutils().notebook().getContext().notebookPath().get()

# Get the repo's root directory name.

repo_root = os.path.dirname(os.path.dirname(notebook_path))

print(repo_root)

# # Prepare to run pytest from the repo.

os.chdir(f"/Workspace{repo_root}")

print(os.getcwd())

# # Skip writing pyc files on a readonly filesystem.

sys.dont_write_bytecode = True

# # Run pytest.

retcode = pytest.main([".", "-v", "-p", "no:cacheprovider"])

# # Fail the cell execution if there are any test failures.

assert retcode == 0, "The pytest invocation failed. See the log for details."

----------------------------

But i am getting below error .

file /Workspace/Repos/sudhanshu.raj@zuhlke.com/databricks_test_practice/tests/test_file.py, line 4 def test_name(): E fixture 'name' not found > available fixtures: capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, monkeypatch, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory > use 'pytest --fixtures [testpath]' for help on them.

Kaniz · ‎09-19-2023

Hi @sudhanshu1 , The error you're encountering is due to the fact that pytest is looking for a fixture named 'name' which is not defined.

Fixtures in pytest are functions that are run before each test function to which it is applied. Fixtures are used to feed some data to the tests such as database connections, URLs to test and some sort of input data.

Therefore, pytest is looking for a fixture 'name' to use with your test function test_name().

In your case, you're not using any fixture, so the error seems to be a misunderstanding. If you're not intending to use a fixture, then the test function test_name() should not have any fixture named 'name'.

To resolve this issue, ensure that your test function does not require any fixture or if it does, make sure the fixture is defined.

sudhanshu1 · ‎09-19-2023

Thanks Kaniz or your answer. However I think i am struggling to understand some basics.

Currently i just wrote a simple test function.

def test_something():

assert 1==1

This test passes , but when i make any small change in this function ,it doesnt reflect .

Example i change assert to 1==0, but my output is still same.

How to refresh main notebook which i am using to run tests files to pick latest change from tests files please? One solution is to restart cluster every time , but that's very bad solution