A Comprehensive Diagnostic Framework for Evaluating Business Intelligence and Analytics Effectiveness

Abstract

Business intelligence and analytics (BIA) initiatives are costly, complex and experience high failure rates. Organizations require effective approaches to evaluate their BIA capabilities in order to develop strategies for their evolution. In this paper, we employ a design science paradigm to develop a comprehensive BIA effectiveness diagnostic (BIAED) framework that can be easily operationalized. We propose that a useful BIAED framework must assess the correct factors, should be deployed in the proper process context and acquire the appropriate input from different constituencies within an organization. Drawing on the BIAED framework, we further develop an online diagnostic toolkit that includes a comprehensive survey instrument. We subsequently deploy the diagnostic mechanism within three large organizations in North America (involving over 1500 participants) and use the results to inform BIA strategy formulation. Feedback from participating organizations indicates that BIA diagnostic toolkit provides insights that are essential inputs to strategy development. This work addresses a significant research gap in the area of BIA effectiveness assessment.