Errors, Irregularities, and Misdirection: Cue Utilisation and Cognitive Reflection in the Diagnosis of Phishing Emails
DOI:
https://doi.org/10.3127/ajis.v26i0.3615Keywords:
cyber security, phishing, cue utilisation, cognitive reflection, expertiseAbstract
The study aimed to examine the role of, and potential interplay between, cue utilisation and cognitive reflection in email users’ ability to accurately (and efficiently) differentiate between phishing and genuine emails. 145 participants completed the Cognitive Reflection Test (CRT), a phishing diagnostic task, and the Expert Intensive Skill Evaluation (EXPERTise 2.0) battery, which provided a gauge of users’ cue utilisation in the domain. The results revealed an interaction between users’ cognitive utilisation and cue reflection, whereby users low in both facets performed significantly worse in diagnosing phishing emails than all other groups. Further, those participants with both higher cue utilisation and cognitive reflection took significantly longer to make their diagnosis. It is concluded that a high level of cognitive reflection was able to compensate for a lower level of cue utilisation, and vice versa. Participants reported using seven types of cue during diagnosis, however, there was no significant relationship between the types of cues used and users’ level of cue utilisation. Taken together, the findings have implications to the design of user-level interventions in relation to the identification of vulnerable users, as well as the need to consider training approaches that extend beyond the use of simple cue inventories.
References
Anderson, J. R. (1996). ACT: A simple theory of complex cognition. American psychologist, 51(4), 355-365. https://doi:10.1037/0003-066X.51.4.355
Basnet R.B., Sung A.H., & Liu Q. (2012) Feature Selection for Improved Phishing Detection. In: Jiang H., Ding W., Ali M., & Wu X. (eds) Advanced Research in Applied Artificial Intelligence. IEA/AIE 2012. Lecture Notes in Computer Science, vol 7345. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-310
Bayl-Smith, P., Sturman, D., & Wiggins, M. (2020). Cue utilization, phishing feature and phishing email detection. In M. Bernhard, A. Bracciali, L. J. Camp, S. Matsuo, A. Maurushat, P. B. Rønne, & M. Sala (Eds.), Financial Cryptography and Data Security: FC 2020 International Workshops, AsiaUSEC, CoDeFi, VOTING, and WTSC, Revised Selected Papers (pp. 56-70). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12063). Springer. https://doi.org/10.1007/978-3-030-54455-3_5
Bengtsson, M. (2016). How to plan and perform a qualitative study using content analysis. Nursing Plus Open, 2, 8-14. https/doi:10.1016/j.npls.2016.01.001
Brams, S., Ziv, G., Levin, O., Spitz, J., Wagemans, J., Williams, A. M., & Helsen, W. F. (2019). The relationship between gaze behavior, expertise, and performance: A systematic review. Psychological Bulletin, 145(10), 980–1027. https://doi.org/10.1037/bul0000207
Brouwers, S., Wiggins, M., & Griffin, B. (2018). Operators who readily acquire patterns and cues, risk being miscued in routinized settings. Journal of Experimental Psychology: Applied, 24(2), 261-274. https://doi:10.1037/xap0000151
Butavicius, M., Parsons, K., Pattison M., & McCormac, A. (2016). Breaching the Human Firewall: Social engineering inPphishing and Spear-Phishing Emails. ArXiv, abs/1606.00887.
Canfield, C. I., Fischhoff, B., & Davis, A. (2016). Quantifying Phishing Susceptibility for Detection and Behavior Decisions. Human Factors: The Journal of the Human Factors and Ergonomics Society, 58(8), 1158–1172. https://doi.org/10.1177/0018720816665025
deGroot, A. D. (1978). Thought and choice in chess. The Hague: Mouton
Downs, J. S., Holbrook, M. B., & Cranor, L. F. (2006). Decision strategies and susceptibility to phishing. In Proceedings of the second symposium on Usable privacy and security (pp. 79-90). ACM. https://doi:10.1145/1143120.1143131
Fette, I., Sadeh, N., & Tomasic, A. (2007). Learning to detect phishing emails. In WWW ‘07 Proceedings of the 16th international conference on World Wide Web (pp. 649-656). New York: ACM. https://doi:10.21236/ada456046
Field, A. (2013). Discovering statistics using IBM SPSS statistics. London, England: Sage.
Fleiss, J. L., Levin, B., & Paik, M. C. (2013). Statistical methods for rates and proportions. Hoboken: John Wiley & Sons.
Frauenstein, E. D., & Flowerday, S. (2020). Susceptibility to phishing on social network sites: A personality information processing model. Computers & Security, 94, 101862. https://doi.org/10.1016/j.cose.2020.101862
Frederick, S. (2005). Cognitive Reflection and Decision Making. Journal of Economic Perspectives, 19(4), 25-42. https://doi.org/10.1257/089533005775196732
French, K. E., & Nevett, M. E. (1993). The Development of Expertise in Youth Sport. In J. L. Starkes & F. Allard (Eds.), Advances in Psychology (pp. 255-270). North-Holland. https://doi.org/https://doi.org/10.1016/S0166-4115(08)61475-2
Gacasan, E. M. P., Wiggins, M. W., & Searle, B. J. (2016). The role of cues in expert project manager sensemaking. Construction Management and Economics, 34(7-8), 492-507. https://doi:10.1080/01446193.2016.1177190
Galanter, C. A., & Patel, V. L. (2005) Medical decision making: A selective review for child psychiatrists and psychologists. Journal of Child Psychology and Psychiatry, 46(7), 675-689. https://doi:10.1111/j.1469-7610.2005.01452.x
Gonzalez, C. (2013). The boundaries of instance-based learning theory for explaining decisions from experience. In V. S. C. Pammi & N. Srinivasan (Eds.), Progress in brain research (Vol. 202, pp. 73-98). Oxford, UK: Elsevier.
Gwet, K. (2001). Handbook of inter-rater reliability. Gaithersburg: STATAXIS Publishing Company.
Harrison, B., Svetieva, E., & Vishwanath, A. (2016). Individual processing of phishing emails. Online Information Review, 40(2), 265–281. https://doi.org/10.1108/oir-04-2015-0106
Harré, M., Bossomaier, T., & Snyder, A. (2012). The perceptual cues that reshape expert reasoning. Scientific Reports, 2(1), 502–502. https://doi.org/10.1038/srep00502
Hassandoust, F., Singh, H., & Williams, J. (2020). The Role of Contextualization in Individuals’ Vulnerability to Phishing Attempts. Australasian Journal of Information Systems, 24. DOI: https://doi.org/10.3127/ajis.v24i0.2693
Johnston, D., & Morrison, B. W. (2016). The application of naturalistic decision-making techniques to explore cue use in rugby league playmakers. Journal of Cognitive Engineering and Decision Making, 10(4), 391-410. https://doi:10.1177/1555343416662181
Jones, H.S., Towse, J. N., & Race, N. (2015) Susceptibility to email fraud: A review of psychological perspectives, data-collection methods, and ethical considerations. International Journal of Cyber Behaviour, Psychology and Learning, 5(3). 13-29. https://doi:10.4018/IJCBPL.2015070102
Jones, H.S., Towse, J. N., Race, N., & Harrison, T. (2019). Email fraud: The search for psychological predictors of susceptibility. PloS ONE 14(1), e0209684. https://doi:10.1371/journal.pone.0209684
Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American psychologist, 64(6), 515-526. https://doi:10.1037/a0016755
Klein, G. A. (2008). Naturalistic Decision Making. Human Factors, 50(3), 456–460. https://doi.org/10.1518/001872008X288385
Klein, G. A., Calderwood, R., & Clinton-Cirocco, A. (1986). Rapid Decision making on the Fire Ground. Proceedings of the Human Factors Society Annual Meeting, 30(6), 576-580.
Kobus, D. A., Proctor, S., & Bank, T. E. (2000). Decision-making in a dynamic environment: the effects of experience and information uncertainty. Technical Report 1832. San Diego, CA: Spawar Systems Center.
Levine, T. R. (2014). Truth-Default Theory (TDT). Journal of Language and Social Psychology, 33(4), 378–392. https://doi:10.1177/0261927x14535916
Loveday, T., Wiggins, M., Festa, M., Schell D., & Twigg, D. (2013). Pattern recognition as an indicator of diagnostic expertise. In C. P. Latorre & F. A. Sanchez (Eds.), Pattern recognition – Applications and methods (pp. 1-11). Berlin: Springer.
Luo, X., Zhang, W., Burd, S., & Seazzu, A. (2013). Investigating phishing victimization with the Heuristic-systematic model: a theoretical framework and an exploration. Computer Security, 38, 28–38. https://doi.org/10.1016/j.cose.2012.12.003
Moghimi, M., & Varjani, A. Y. (2016). New rule-based phishing detection method. Expert Systems with Applications, 53, 231–242. https://doi:10.1016/j.eswa.2016.01.028
Morrison, B. W., Johnston, D., Naylor, M., Morrison, N. M. V., & Forrest, D. (2020). “You can’t hide your lyin’ eyes”: investigating the relationship between associative learning, cue awareness, and decision performance in detecting lies. Journal of Cognitive Engineering and Decision Making, 14(2), 99-111. https://doi.org/10.1177/1555343420918084
Morrison, B., & Morrison, N. (2015). Diagnostic cues in major crime investigation. In M. W. Wiggins, & T. Loveday (Eds.), Diagnostic expertise in organizational environments (pp. 91-98). Ashgate Publishing
Morrison, B. M., Wiggins, M. W., Bond N. W., & Tyler, M. D. (2013). Measuring relative cue strength as a means of validating an inventory of expert offender profiling cues. Journal of Cognitive Engineering and Decision Making, 7(2), 211-226. https://doi:1177/
Morrison, B. W., Wiggins , M. W., & Morrison, N. V. (2018). Utility of expert cue exposure as a mechanism to improve decision-making performance among novice criminal investigators. Journal of Cognitive Engineering and Decision Making, 12(2), 99-111. https://doi:10.1177/1555343417746570
Nasser, G., Morrison, B. W., Bayl-Smith, P., Taib, R., Gayed, M., & Wiggins, M. W. (2020a). The effects of cue utilization and cognitive load in the detection of phishing emails. In AsiaUSEC’20: proceedings of the Workshop on Usable Security (pp. 1-10). Malaysia: Springer.
Nasser, G., Morrison, B. W., Bayl-Smith, P., Taib, R., Gayed, M., & Wiggins, M. W. (2020b). The Role of Cue Utilization and Cognitive Load in the Recognition of Phishing Emails. Frontiers in big data, 3, 546860. https://doi.org/10.3389/fdata.2020.546860
Parsons, K., Butavicius, M., Pattinson, M., McCormac, A., Calic, D., & Jerram, C. (2016). Do users focus on the correct cues to differentiate between phishing and genuine emails? arXiv preprint arXiv:1605.04717.
Qualtrics (Version 2019). [Web-based software]. Provo, UT: Qualtrics. Available from http://www.qualtrics.com.
Shanteau, J., Weiss, D. J., Thomas, R. P., & Pounds, J. C. (2002). Performance-based assessment of expertise: How to decide if someone is an expert or not. European Journal of Operational Research, 136(2), 253-263. https://doi:10.1016/S0377-2217(01)00113-8
Shekh, S., Auton, J. C., & Wiggins, M. W. (2018). The effects of cue utilization and target-related information on target detection during a simulated drone search and rescue task. Proceedings of the Human Factor and Ergonomics Society Annual Meeting, 62(1), 227-231. https://doi:10.1177/1541931218621053
Shonman, M., Li, X., Zhang, H., & Dahbura, A. (2018). Simulating phishing email processing with instance-based learning and cognitive chunk activation. In S. Wang, V. Yamamoto, J. Su, Y. Yang, E. Jones, L Iasemidis & T. Mitchell (Eds.), Lecture Notes in Computer Science: Vol 11309. Brain Informatics (pp. 468-478). Cham: Springer. https://doi:10.1007/978-3-030-05587-5_44
Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate? Behavioral and Brain Sciences, 23(5), 645-665. https://doi:10.1017/
S0140525X00003435
Kahneman, D., & Tversky, A. (1972). Subjective probability: A judgment of representativeness. Cognitive Psychology, 3(3), 430-454. https://doi.org/10.1016/0010-0285(72)90016-3.
Tversky, A., & Kahneman, D. (1974). Heuristics and biases: Judgement under uncertainty. Science, 185(1974), 1124-1130. https://doi:10.1126/science.185.4157.1124
Tversky A., & Kahneman D. (1975) Judgment under Uncertainty: Heuristics and Biases. In: Wendt D., Vlek C. (eds) Utility, Probability, and Human Decision Making. Theory and Decision Library (An International Series in the Philosophy and Methodology of the Social and Behavioral Sciences), vol 11. Springer, Dordrecht. https://doi.org/10.1007/978-94-010-1834-0_8
Vishwanath, A. (2015). Habitual Facebook Use and its Impact on Getting Deceived on Social Media. Journal of Computer-Mediated Communication, 20(1), 83-98
Vishwanath, A., Harrison, B., & Ng, Y.J. (2016). Suspicion, cognition, and automaticity model of phishing susceptibility. Commun. Res. 1–21. https://doi.org/10.1177/0093650215627483
Vishwanath, A., Harrison, B., & Ng, Y. J. (2018). Suspicion, Cognition, and Automaticity Model of Phishing Susceptibility. Communication Research, 45(8), 1146–1166. https://doi.org/
1177/0093650215627483
Vishwanath, A., Herath, T., Chen, R., Wang, J., & Rao, H.R. (2011), “Why do people get phished? Testing individual differences in phishing vulnerability within an integrated information processing model”. Decision Support Systems, 51(3), 576-586.
Walczyk, J. J., Mahoney, K. T., Doverspike, D., & Griffith-Ross, D. A. (2009). Cognitive lie detection: Response time and consistency of answers as cues to deception. Journal of Business and Psychology, 24, 33-49. https://doi:10.1007/s10869-009-9090-8
Watkinson, J., Bristow, G., Auton, J., McMahon, C. M., & Wiggins, M. W. (2018). Postgraduate training in audiology improves clinicians’ audiology-related cue utilisation. International Journal of Audiology, 57(9), 681-687. https://doi:10.1080/14992027.2018.1476782
Weiss, D. J., & Shanteau, J. (2003). Empirical assessment of expertise. Human Factors: The Journal of the Human Factors and Ergonomics Society, 45(1), 104-116. https://doi:10.1518/hfes.45.1.
27233
Wickens, C. D., Hollands, J. G., Banbury, S., & Parasuraman, R. (2013). Engineering psychology and human performance. New York: Psychology Press. https://doi:10.4324/9781315665177
Wiggins, M. W. (2021). A behaviour-based approach to the assessment of cue utilisation: implications for situation assessment and performance. Theoretical Issues in Ergonomics Science, 22(1), 46-62. https://doi.org/10.1080/1463922X.2020.1758828
Wiggins, M. W. (2016). Expertise and cognitive skills development for ab-initio pilots. In R. A. Telfer & P. J. Moore (Eds.), Aviation training: Learners, instruction and organization (pp 54-66). Abington, Oxon: Routledge.
Wiggins, M. W., Brouwers, S., Davies, J., & Loveday, T. (2014). Trait-based cue utilization and initial skill acquisition: implications for models of the progression to expertise. Frontiers in Psychology, 5, 541. https://doi:10.3389/fpsyg.2014.00541
Wiggins, M. W., Crane, M., & Loveday, T. (2018). Cue utilization, perceptions, and experience in the interpretation of weather radar returns. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 62(1), 721-725. https://doi:10.1177/1541931218621164
Wiggins, M, W., Griffins, B., & Brouwers, S. (2019). The potential role of context-related exposure in explaining differences in water safety cue utilization. Human Factors: The Journal of the Human Factors and Ergonomics Society, 61(5), 825-838. https://doi:10.1177/
Wiggins, M., Loveday, T., & Lyons, L. (2014). Cues and cue-based processing: Implications for system safety. Procedia Engineering, 84, 55-61. https://doi:10.1016/j.proeng.2014.10.409
Wiggins, M. W., Whincup, E., & Auton, J. C. (2018). Cue utilisation reduces effort but increases arousal during a process control task. Applied Ergonomics, 69, 120-127. https://doi:10.1016/
j.apergo.2018.01.012
Williams, E. J., Hinds, J., & Joinson, A. N. (2018). Exploring susceptibility to phishing in the workplace. International Journal of Human-Computer Studies, 120, 1-13. https://doi:10.1016/
j.ijhcs.2018.06.004
Yan, Z., & Gozu, H. Y. (2012). Online decision-making in receiving spam emails among college students. International Journal of Cyber Behavior, Psychology and Learning, 2(1), 1-12. https://doi:10.4018/ijcbpl.2012010101
Downloads
Published
How to Cite
Issue
Section
License
AJIS publishes open-access articles distributed under the terms of a Creative Commons Non-Commercial and Attribution License which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and AJIS are credited. All other rights including granting permissions beyond those in the above license remain the property of the author(s).
