Machine Learning Based Decision-Making: A Sensemaking Perspective

Authors

  • Jingqi (Celeste) Li The University of Queensland
  • Morteza Namvar The University of Queensland
  • Ghiyoung P. Im University of Louisville
  • Saeed Akhlaghpour The University of Queensland

DOI:

https://doi.org/10.3127/ajis.v28.4781

Keywords:

Decision-making, Sensemaking, Machine Learning (ML)

Abstract

The integration of machine learning (ML), functioning as the core of various artificial intelligence (AI)-enabled systems in organizations, comes with the assertion that ML models offer automated decisions or assist domain experts in refining their decision-making. The current research presents substantial evidence of ML’s positive impact on business and organizational performance. Nonetheless, there is a limited understanding of how decision-makers participate in the process of generating ML-driven insights and enhancing their comprehension of business environments through ML outcomes. To enhance this engagement and understanding, this study examines the interactive process between decision-makers and ML experts as they strive to comprehend an environment and gather business insights for decision-making. It builds upon Weick’s sensemaking model by integrating ML’s pivotal role. By conducting interviews with 31 ML experts and ML end-users, we explore the dimensions of sensemaking in the context of ML utilization for decision-making. Consequently, this study proposes a process model which advances the organizational ML research by operationalizing Weick’s work into a structured ML-driven sensemaking model. This model charts a pragmatic pathway, outlining the interaction sequence between decision-makers and ML tools as they navigate through recognizing and utilizing ML, exploring opportunities, assessing ML model outcomes, and translating ML models into action, thereby advancing both the theoretical framework and its practical deployment in organizational contexts.

References

Baird, A., & Maruping, L. M. (2021). The next generation of research on IS use: A theoretical framework of delegation to and from agentic IS artifacts. MIS Quarterly, 45(1), 315–341.

https://doi.org/10.25300/MISQ/2021/15882

Baker, J., Jones, D., & Burkman, J. (2009). Using visual representations of data to enhance sensemaking in data exploration tasks. Journal of the Association for Information Systems, 10, 533–559. https://doi.org/10.17705/1jais.00204

Bhimani, A. (2015). Exploring big data’s strategic consequences. Journal of Information Technology, 30(1), 66–69. https://doi.org/10.1057/jit.2014.29

Boland, R. (2008). Decision making and sensemaking. In Handbook on Decision Support Systems (pp. 55–63). Berlin, Heidelberg, Germany: Springer Publishing.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.

https://doi.org/10.1191/1478088706qp063oa

Chen, H., Chiang, R. H. L., & Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS Quarterly, 36(4), 1165–1188.

https://doi.org/10.2307/41703503

Collins, C., Dennehy, D., Conboy, K., & Mikalef, P. (2021). Artificial intelligence in information systems research: A systematic literature review and research agenda. International Journal of Information Management, 60, Article 102383.

https://doi.org/10.1016/j.ijinfomgt.2021.102383

Conboy, K., Fitzgerald, G., & Mathiassen, L. (2012). Qualitative methods research in information systems: Motivations, themes, and contributions. European Journal of Information Systems, 21(2), 113–118. https://doi.org/10.1057/ejis.2011.57

Cui, R., Li, M., & Zhang, S. (2022). AI and procurement. Manufacturing & Service Operations Management, 24(2), 691–706.

https://doi.org/10.1287/msom.2021.0989

de Laat, P. B. (2018). Algorithmic decision-making based on machine learning from big data: Can transparency restore accountability? Philosophy & Technology, 31(4), 525–541.

https://doi.org/10.1007/s13347-017-0293-z

Deng, X., & Chi, L. (2012). Understanding postadoptive behaviors in information systems use: A longitudinal analysis of system use problems in the business intelligence context. Journal of Management Information Systems, 29(3), 291–326.

https://doi.org/10.2753/MIS0742-1222290309

Dickson, S. A. (2015). Stuck in the middle: Generation Xers lived experiences in the workforce while navigating between baby boomers and millennials, and perceived impact on their job satisfaction. [Doctoral Dissertation, Colorado State University]. Mountain Scholar. https://api.mountainscholar.org/server/api/core/bitstreams/27b6c9fd-3f9e-4bd9-8a14-bb402d7d6d0f/content

Dinter, B. (2013). Success factors for information logistics strategy—An empirical investigation. Decision Support Systems, 54(3), 1207–1218.

https://doi.org/10.1016/j.dss.2012.09.001

Enholm, I. M., Papagiannidis, E., Mikalef, P., & Krogstie, J. (2021). Artificial intelligence and business value: A literature review. Information Systems Frontiers. 24(5), 1709–1734.

https://doi.org/10.1007/s10796-021-10186-w

Fügener, A., Grahl, J., Gupta, A., & Ketter, W. (2021). Will humans-in-the-loop become borgs? merits and pitfalls of working with AI. MIS Quarterly, 45(3), 1527–1556.

https://doi.org/10.25300/MISQ/2021/16553

Ghasemaghaei, M. (2019). Does data analytics use improve firm decision making quality? The role of knowledge sharing and data analytics competency. Decision Support Systems. 120, 14–24. https://doi.org/10.1016/j.dss.2019.03.004

Ghasemaghaei, M., Ebrahimi, S., & Hassanein, K. (2017). Data analytics competency for improving firm decision making performance. The Journal of Strategic Information Systems, 27(1), 101–113. https://doi.org/10.1016/j.jsis.2017.10.001

Gioia, D. A., Corley, K. G., & Hamilton, A. L. (2013). Seeking qualitative rigor in inductive research: Notes on the Gioia methodology. Organizational Research Methods, 16(1), 15–31.

https://doi.org/10.1177/1094428112452151

Grover, V., Chiang, R. H. L., Liang, T.-P., & Zhang, D. (2018). Creating strategic business value from big data analytics: A research framework. Journal of Management Information Systems, 35(2), 388–423. https://doi.org/10.1080/07421222.2018.1451951

Günther, W. A., Mehrizi, M. H. R., Huysman, M., & Feldberg, F. (2017). Debating big data: A literature review on realizing value from big data. The Journal of Strategic Information Systems, 26(3), 191–209. https://doi.org/10.1016/j.jsis.2017.07.003

Hasan, H., & Gould, E. (2001). Support for the sense-making activity of managers. Decision Support Systems, 31(1), 71–86. https://doi.org/10.1016/S0167-9236(00)00120-2

Hekkala, R., Stein, M.-K., & Rossi, M. (2018). Metaphors in managerial and employee sensemaking in an information systems project. Information Systems Journal, 28(1), 142–174. https://doi.org/10.1111/isj.12133

Jennings, P. D., & Greenwood, R. (2003). Constructing the iron cage: Institutional theory and enactment. In R. Westwood & S. Clegg (Eds.), Debating organization: Point-counterpoint in organization studies (pp.195–207), Hoboken, NJ, USA: Wiley-Blackwell.

Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260. https://doi.org/10.1126/science.aaa8415

Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64(6), 515–526. https://doi.org/10.1037/a0016755

KhakAbi, S., Gholamian, M. R., & Namvar, M. (2010, January 27–29). Data Mining Applications in Customer Churn Management. 2010 International Conference on Intelligent Systems, Modelling and Simulation. Liverpool, United Kingdom.

https://doi.org/10.1109/ISMS.2010.49

Krishnamoorthi, S., & Mathew, S. K. (2018). Business analytics and business value: A comparative case study. Information & Management, 55(5), 643–666.

https://doi.org/10.1016/j.im.2018.01.005

Li, M., & Li, T. (2022). AI automation and retailer regret in supply chains. Production and Operations Management, 31(1), 83–97. https://doi.org/10.1111/poms.13498

Li, X., Po-An Hsieh, J. J., & Rai, A. (2013). Motivational differences across post-acceptance information system usage behaviors: An investigation in the business intelligence systems context. Information Systems Research, 24(3), 659–682.

https://doi.org/10.1287/isre.1120.0456

Ligon, J., & Sim, J.-S. (2000). Which companies are more profitable when using data mining techniques? Americas Conference on Information Systems, Long Beach, California, August 10–13, 2000.

http://aisel.aisnet.org/amcis2000/381?utm_source=aisel.aisnet.org%2Famcis2000%2F381&utm_medium=PDF&utm_campaign=PDFCoverPages.

Maitlis, S., & Christianson, M. (2014). Sensemaking in Organizations: Taking Stock and Moving Forward. The Academy of Management Annals. The Academy of Management Annals, 8(1), 57–125. https://doi.org/10.1080/19416520.2014.873177

Manyika, J., Lund, S., Chui, M., Bughin, J., Woetzel, J., Batra, P., Ko, R., & Sanghvi, S. (2017). Jobs lost, jobs gained: Workforce transitions in a time of automation. https://epic.org/wp-content/uploads/foia/epic-v-ai-commission/EPIC-19-09-11-NSCAI-FOIA-20200529-5th-Production-pt4-Outside-Reports-Resources.pdf [Accessed March 13, 2022].

Namvar, M., & Cybulski, J. (2014). BI-based organizations: A sensemaking perspective. International Conference on Information Systems, Auckland, New Zealand, December 14–17. https://aisel.aisnet.org/icis2014/proceedings/ISStrategy/39.

Namvar, M., Cybulski, J., Phang, C., Wee, C.-Y., & Tan, K. Y. (2018). Simplifying sensemaking: Concept, process, strengths, shortcomings, and ways forward for information systems in contemporary business environments. Australasian Journal of Information Systems, 22. https://doi.org/10.3127/ajis.v22i0.1654

Namvar, M., Intezari, A., Akhlaghpour, S., & Brienza, J. P. (2022). Beyond effective use: Integrating wise reasoning in machine learning development. International Journal of Information Management, 69, Article 102566.

https://doi.org/10.1016/j.ijinfomgt.2022.102566

Newman, M., & Robey, D. (1992). A social process model of user-analyst relationships. MIS Quarterly, 16(2), 249–266. https://doi.org/10.2307/249578

Seidel, S., Kruse, L. C., Székely, N., Gau, M., & Stieger, D. (2018). Design principles for sensemaking support systems in environmental sustainability transformations. European Journal of Information Systems, 27(2), 221–247.

https://doi.org/10.1057/s41303-017-0039-0

Sharma, R., Mithas, S., & Kankanhalli, A. (2014). Transforming decision-making processes: A research agenda for understanding the impact of business analytics on organisations. European Journal of Information Systems, 23(4) 433–441.

https://doi.org/10.1057/ejis.2014.17

Shollo, A., & Galliers, R. D. (2016). Towards an understanding of the role of business intelligence systems in organisational knowing. Information Systems Journal, 26(4), 339–367. https://doi.org/10.1111/isj.12071

Simester, D., Timoshenko, A., & Zoumpoulis, S. I. (2020). Targeting prospective customers: Robustness of machine-learning methods to typical data challenges. Management Science, 66(6), 2495–2522. https://doi.org/10.1287/mnsc.2019.3308

Stetson, P. D., Bakken, S., Wrenn, J. O., & Siegler, E. L. (2012). Assessing electronic note quality using the physician documentation quality instrument (PDQI-9). Applied Clinical Informatics, 3(02), 164–174. https://doi.org/10.4338/ACI-2011-11-RA-0070

Surbakti, F. P. S., Wang, W., Indulska, M., & Sadiq, S. (2019). Factors Influencing Effective Use of Big Data: A Research Framework. Information & Management, 57(1), Article 103146.

https://doi.org/10.1016/j.im.2019.02.001

Teodorescu, M. H., Morse, L., Awwad, Y., & Kane, G. C. (2021). Failures of fairness in automation require a deeper understanding of human-ML augmentation. MIS Quarterly, 45(3), 1483–1499. https://doi.org/10.25300/MISQ/2021/16535

Torres, R., Sidorova, A., & Jones, M. C. (2018). Enabling firm performance through business intelligence and analytics: A dynamic capabilities perspective. Information & Management, 55(7), 822–839. https://doi.org/10.1016/j.im.2018.03.010

Trieu, V.-H. (2017). Getting value from Business Intelligence systems: A review and research agenda. Decision Support Systems, 93, 111–124. https://doi.org/10.1016/j.dss.2016.09.019

van den Broek, E., Sergeeva, A., & Huysman, M. (2021). When the machine meets the expert: An ethnography of developing AI for hiring. MIS Quarterly, 45(3), 1557–1580.

https://doi.org/10.25300/MISQ/2021/16559

Weick, K. E. (1979). The Social Psychology of Organizing. (2nd edn.). New York, USA: McGraw-Hill.

Weick, K. E. (1995). Sensemaking in organizations (Vol. 3). Thousand Oaks, CA, USA: Sage Publications.

Weick, K. E., Sutcliffe, K. M., & Obstfeld, D. (2005). Organizing and the process of sensemaking. Organization Science, 16, 409–421.

Woerner, S. L., & Wixom, B. H. (2015). Big data: Extending the business strategy toolbox. Journal of Information Technology, 30(1), 60–62. https://doi.org/10.1057/jit.2014.31

Woodside, A. G. (2005). Opening up decision making: Making sense of entrepreneur and reseller business-to-business strategies. Journal of Business & Industrial Marketing, 20(7), 347–354. https://doi.org/10.1108/08858620510628588

Wu, W., Chen, J., Yang, Z., & Tindall, M. L. (2021). A cross-sectional machine learning approach for hedge fund return prediction and selection. Management Science, 67(7), 4577–4601. https://doi.org/10.1287/mnsc.2020.3696

Yoo, Y. (2015). It is not about size: A further thought on big data. Journal of Information Technology, 30(1), 63–65. https://doi.org/10.1057/jit.2014.30

Downloads

Published

2024-05-15

How to Cite

Li, J. (Celeste), Namvar, M., Im, G. P., & Akhlaghpour , S. (2024). Machine Learning Based Decision-Making: A Sensemaking Perspective . Australasian Journal of Information Systems, 28. https://doi.org/10.3127/ajis.v28.4781

Issue

Section

Research Articles