cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Certifications
Join dynamic discussions on Databricks certifications within the Community. Exchange insights, tips, and experiences to help prepare for certification exams and validate your expertise in data engineering, analytics, and machine learning.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

DevOps Fundamentals: Lecture - Introduction to Unit Tests for PySpark

Daniel_B
Contributor II

This sentence can be improved as we are not testing unit tests we are writing unit tests:

There are a variety of other methods to test your unit tests, we will focus on the pyspark testing utils.

2 REPLIES 2

Daniel_B
Contributor II

Later, same issue:

There are a variety of other methods to test your unit tests, we will focus on the pyspark testing utils.

Daniel_B
Contributor II

This reads oddly: Next like talk about the pytest unit testing framework.

Uninformative repetition: 

- Just write test functions that start with test_, and pytest will automatically pick them up and run them.

- Just name your test files and functions with the test_ prefix, and pytest will take care of the rest.

Full text from SCORM module for context:

Next like talk about the pytest unit testing framework.
Pytest is a popular testing framework for Python that makes it easy to write simple and scalable test cases. It provides a variety of benefits for executing your unit tests.
First up, pytest has a very simple syntax. You donโ€™t need to worry about complex setup. Just write test functions that start with test_, and pytest will automatically pick them up and run them.
Next, pytest leverages Pythonโ€™s built-in assert statements (or you can use other assert statements like in pyspark). These are simple to use but provide detailed error messages when a test fails, making your function debugging much easier and faster by understanding what went wrong.
Pytest also automatically discovers and runs all your tests. Thereโ€™s no need to manually configure which tests to run. Just name your test files and functions with the test_ prefix, and pytest will take care of the rest.
Lastly, pytest has a rich ecosystem of plugins to extend its functionality. Whether you need test coverage reports, parallel test execution, or integration with other tools, there's a plugin for almost anything, allowing you to customize pytest to fit your needs.
This course provides a simple introduction to pytest. There are many testing frameworks available, select the one that best meets your organization's needs.