topic Azure Databricks Python Test Coverage report in Data Engineering

Azure Databricks Python Test Coverage report

Nilave — Wed, 24 Nov 2021 12:08:50 GMT

Hi,

I'm trying to enable coverage report for Azure Databricks python code. Found a solution using pytest with databricks connect but faced below challenges

Pytest framework using Databricks Connect:- This works for simple notebooks. But doesnt work for below
1. MAGIC commands doesnt work - %run, %sh doesnt works in databricks connect . Ref python 3.x - Execute databricks magic command from PyCharm IDE - Stack Overflow
DButils.widget doesnt work in Databricks connect - Ref - DBUtils cannot find widgets [Windows 10] (databricks.com)

Is there any solution available for getting a coverage report without making much changes in the existing source notebooks.

Re: Azure Databricks Python Test Coverage report

Prabakar — Wed, 24 Nov 2021 12:14:27 GMT

Hi @Nilave Chakraborty I understand the pain in making such changes. But unfortunately, we do not have a solution for now.

Re: Azure Databricks Python Test Coverage report

Nilave — Wed, 08 Dec 2021 10:05:11 GMT

@Prabakar Ammeappin Thanks for the reply. May I know is there any plan for adding code coverage functionality to Databricks python notebooks? If so, any timelines if you can provide that would be great.

Re: Azure Databricks Python Test Coverage report

Prabakar — Wed, 08 Dec 2021 14:39:25 GMT

@Nilave Chakraborty , There is no support for code coverage functionality yet. I don't think it will be available in the near future. However, it will be considered later. I do not have any timeline for now.

Re: Azure Databricks Python Test Coverage report

bharani — Mon, 21 Nov 2022 08:57:04 GMT

Hi Prabakar, I would like to do test coverage in databricks notebooks so is there any way that I can do and also can I do testing with pytest in databricks? Are there any other tests like nutter library and doc test supports coverage?