Authentication is a critical security feature for confirming the identity of a system’s users, typically implemented with help from frameworks like Spring Security. It’s important to robustly test complex security features, and unit testing is an effective technique for verifying the correctness of the fine-grained behaviors of a feature. Unfortunately, resources to help developers unit test security features are limited. Most guides focus on black box or penetration testing, tests produced by existing test generation tools are difficult to maintain, and recommendation solutions usually focus on prioritization and selection of existing tests. These solutions are not applicable to developers writing new unit tests, or who want to use metrics other than coverage to track what has been tested.

In this paper, we address these issues using a grounded theory-based approach to identify common test cases for token authentication by analyzing 481 JUnit tests exercising Spring Security-based authentication implementations from 53 open source Java projects. The outcome of this study is a unit testing guide organized as a catalog of 52 test cases for token authentication, representing unique combinations of 17 scenarios, 40 conditions, and 30 expected outcomes learned from the data set in our analysis. We supplement the test guide with common test smells to avoid. To verify the accuracy and usefulness of our testing guide, we sought feedback from selected developers, some of whom authored unit tests in our dataset.