Showing results for 
Search instead for 
Did you mean: 

Testing framework using Databricks Notebook and Pytest.

Not applicable

Hi Friends,

I am designing a Testing framework using Databricks and pytest. Currently stuck with report generation, that is generating blank with only default parameters only .

for ex :-


<testsuite name="pytest" errors="0" failures="0" skipped="0" tests="0" time="0.157" timestamp="2023-04-11T14:30:47.707323" hostname="XXXXX"/>


Looking for any other framework which can solve my Data Testing.


Not applicable

@Vijaya Palreddy​ :

There are several testing frameworks available for data testing that you can consider using with Databricks and Pytest:

  1. Great Expectations: Great Expectations is an open-source framework that provides a simple way to create and maintain automated data tests. It supports a variety of data sources, including Databricks, and provides customizable reports for data quality checks.
  2. Apache Griffin: Apache Griffin is another open-source framework that supports data quality testing for various data sources, including Databricks. It provides customizable reports for data quality checks and supports multiple data formats.
  3. Data testing framework by Databricks: Databricks also offers a data testing framework that can be used with Pytest. It provides data validation tests for data pipelines, ETL jobs, and data lakes. The framework integrates with Databricks Delta and Apache Spark and generates detailed reports on data quality checks.

You can choose the framework that best fits your needs and integrates well with Databricks and Pytest.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.