Poka-Yoke is a term coming from lean manufacturing and is about error-proofing, or preventing an error happening in the first place. A few weeks ago, we were working on fixing those flaky tests. A cause is the bad testing strategy, trying to set mocks on objects held by spring globaltons.
The previous developers had apparently known about this and went through diligently fixing each tests by ensuring that any test setting a mock was reset to its proper form. Unfortunately this knowledge wasn’t passed on to our team, and wasn’t obvious from the intent of the tests. More than that, it was easy to add new tests of this style and forget to reset the objects registered in spring.
Wanting to apply Poka-Yoka to our situation, we wrote something that would check the precondition that there were no side effects from other tests. Our implementation meant adding a @Before method to the base class these classes all shared that would cycle through all objects registered in spring, noting if any of their internal fields had been left mocked. If anything was detected, our assertion failed message contained a message that would explain why it was there and what to do about it. By adding this in, we actually found three existing tests that failed to clean up properly after themselves.
Poka-Yoka is fairly easy. When thinking about applying it, think about a solution that prevents the error happening more than once. Ideally this means that error is unavoidable (in our situation – this would mean changing the testing pattern across all tests), or at least by bare minimum, provide feedback when a person repeats the mistake.
