Question about test protocol logging given/when/then

[amhest]

I am not sure if this is possible or not. I don't see an example. I am very experienced with Spock/groovy but not as much with python. For my system we need to capture a test report that logs given/when/then steps as part of test execution.

Would there be a "best" way to do something like the sample below with Nimoy? In Spock we would say given: counter is not set. For tests with where blocks we would want to be able to insert variable values into the log statements.

my_module_spec.py

from nimoy.specification import Specification

# The class name ends with Spec and extends Specification
class MyFirstSpec(Specification):

    # The feature method is public
    def my_feature_method(self):
        counter = 0
        with given ("counter is not set"): 
            counter == 0
        with when ("counter is incremented"): 
            counter = counter + 1
        with then ("counter has increased by 1"):
            counter == 1

    # This is not a feature method
    def _helper_method(self):
       pass

[noamt]

Hi, and thanks for taking an interest in Nimoy!

Just to make sure I understand you correctly, you'd expect the output for this test case, on top of the normal output, to be:

counter is not set
counter is incremented
counter has increased by 1

and if there were a parameterised where block, you would expect to be able to embed parameters in this string. Not only the parameters defined in the where block but also parameters from the body of the method.

Is that the case? If yes, then it's not currently implemented, but definitely something that I can work on.

[amhest]

Yes that's what I'm looking for with the addition of the keywords:

given: counter is not set
and: .... 
when: counter is incremented
then: counter has increased by 1

I only care about parameter values defined in the where block. The idea is that that the exploded where clause plus given/when/then constitute a "test protocol" document while a log showing this plus the results of assert/expect statements constitutes a "test report" document. These documents showing the tests we plan to run plus the results of actually running the tests in human readable form are commonly required by regulators and auditors for software in regulated environments. (E.g. medical, automotive etc. ) We don't want the "protocol" impacted by the execution so we don't need to accommodate changes in code.

I'm really excited to find Nimoy as I've used Spock for this in the past and find it much better suited than cucumber etc. which require the feature files to be separate from their implementation. While cucumber might seem to make the protocol easier to report, in my experience separate feature files makes tests less efficient for experienced engineers to develop and can be harder to review, especially over time as multiple teams work on similar tests for different modules.

[noamt]

I'm really excited to find Nimoy

I'm happy you like it!

I'm the sole maintainer so I'll do my best to get to this. In any case, contributions are always welcome!

[amhest]

I'm happy to help but would need some pointers on where to start. I can definitely contribute testing help.

[noamt]

So yeah you can definitely PR a bunch of spec for this.
Nimoy is actually dogfooding and all tests are written with Nimoy.
You can start by adding an integration spec which executes your example spec and asserts the output.

I can then guide you towards the implementation of it as well