- 2356 Views
- 1 replies
- 0 kudos
Unit test with Nutter
When I run the simple test in a notebook, it works fine, but when I run it from the Azure ADO pipeline, it fails with the error.code;def __init__(self): NutterFixture.__init__(self) from runtime.nutterfixture import NutterFixture, tagclass uTestsDa...
- 2356 Views
- 1 replies
- 0 kudos
- 0 kudos
@Chris Konsur​ :The error message suggests that there is an issue with the standard output buffer when the Python interpreter is shutting down, which could be related to daemon threads. This error is not specific to Databricks or Azure ADO pipeline, ...
- 0 kudos
- 4303 Views
- 8 replies
- 8 kudos
Using .repartition(100000) causes the unit test to be extremely slow (>20 mins). Is there a way to speed it up?
Here's the code:val result = spark .createDataset(List("test")) .rdd .repartition(100000) .map { _ => "test" } .collect() .toList println(result)I write tests to test for correctness, so I wonde...
- 4303 Views
- 8 replies
- 8 kudos
- 8 kudos
Hey there @tanin​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
- 8 kudos