Recently, I discovered a non-deterministic behavior issue during pair testing with my coworkers, which was related to the enable/disable WhatsApp option that prevents users from receiving any notifications about JSP/ITA1. Non-deterministic behavior is quite different from intermittent issues. Non-deterministic behavior usually occurs with recognizable patterns, but reproducing the issue may require different test scenarios
In this context, we assumed that when an existing user tries to disable their WhatsApp option, there is still a chance that the user might receive an invitation. During our investigation using traffic replay and requests to another API, we found that the WhatsApp subscription remained active even though, based on the predefined state, it had already been disabled
Non-deterministic behavior becomes a common problem when data exchanges are heavily dependent on other services. How do we prevent that? Both in theory and in practice, it’s quite difficult to solve. Even if preventive actions have already been implemented, there are still residual risks and secondary risks that may occur.
- residual risk → the risk that still exists even though controls have been implemented to mitigate or prevent it
- secondary risk → the potential risk that may arise as a consequence of implementing those preventive measures
Footnotes
-
It’s one of our features that allows recruiters to invite particular job seekers to apply. It stands for “Invite to Apply” ↩