How to gain competitive advantages over competitors should be a crucial question for any organization. Organizations that have scaled advanced analytics (predictive analytics, prescriptive analytics) beyond pilot projects and characterize themselves as data-driven tend to be top performers.13 Furthermore, there is no shortage of examples of how advanced analytics can be applied to a given domain.
In 2022, NewVantagePartners released its 10th annual report on data and AI leadership among Fortune 1000 organizations.14 Although progress has been made over the years to become more data-driven, there are three major concerns in the 2022 report: only 19.3% had established a data-driven culture; only 26.5% had established a data-driven organization; and the nontechnical barriers represented 91.9% of the perceived challenges to become more data-driven. In addition, the metrics are declining. One can only speculate whether more respondents in the survey or a more critical self-assessment of the organizations are the reasons for the low figures or if the explanation can be found elsewhere. In practice, it means organizations cannot use advanced analytics to its full potential.
Barriers to becoming a data-driven organization are well known and include:3 lack of understanding, lack of skills, insufficient organizational alignment, lack of management support, and lack of vision and supporting strategies. Our experience indicates organizations take shortcuts when scaling advanced analytics and have a too-strong technical focus with limited use of an overall change management process. Similarly, Davenport and Bean8 suggested "… firms need more concerted programs to achieve data-related cultural change," and must include more experts in organizational change and change management to overcome the barriers.14
The question is whether scaling advanced analytics is any different from scaling the usage of a new tool, business process, or workflow, for example, a new model for inpatient care flow10 or a new sales tracking system.1 The shared findings in Ashkenas and Matta1 and Golden-Biddle10 indicate scaling is possible as long as one does not assume everything from the initial testing of the scaled object (inpatient care flow, sales tracking system) can be replicated. Instead, the scaled object must be adapted to the new environment together with user training. Now assume initial testing of advanced analytics is done for a specific business process in sales. Following the previous scaling recommendations, the specific business process should be scaled and adapted to new environments, and users should receive training. If an organization intends to become data-driven and establish a data-driven culture in teams, the focus of the scaling should be the usage of advanced analytics within the organization, and not how advanced analytics is applied to a specific workflow or business process. Hence, previous recommendations for scaling must be adapted when scaling advanced analytics.
Given the reported problems with scaling advanced analytics, requests for more concerted programs, and increased focus on change management, what are the key elements for a proper scaling of advanced analytics?
How to start. Many organizations have started their journey to become more data-driven with pilot projects in advanced analytics.3 Our experience indicates these pilot projects tend to be fairly straightforward. One reason is they are focused on applying technical aspects to a given dataset, for example, can this data-mining algorithm give us any new insights on our sales data? In case only decision makers and data scientists are involved, the pilot project is too heavily focused on technical aspects since employees working with sales and sales data are bypassed.
Although an initial project on advanced analytics shares many barriers and practices from any type of project, there are at least three barriers that need extra attention:2 data quality; business resistance; and business overconfidence. We have encountered several initial projects in advanced analytics that have been delayed due to poor data quality or where the data has been manipulated (to fit the pilot project). Several initial projects in advanced analytics intend to mimic how a middle manager makes decisions. We have seen an increased level of business resistance among middle managers that do not want to share their knowledge with a data scientist. In contrast, we have also seen evidence of business overconfidence where senior managers make statements about algorithm accuracy that is off the scale.
Regardless if organizations start with a pilot project or apply other mechanisms, for example, work discovery,10 to collect initial experience of using advanced analytics, there are some underpinning best common practices:2,6,7 align the initial work to existing business strategies or important goals within the organization; use a cross-functional team; and document the initial work as a preparation for making a future decision if scaling should be done or not. Hence, any initial work in advanced analytics that is carried out as a big-bang project with limited or no involvement of business users is likely to act as a catalyst for business resistance.
Scaling decision. Once the initial experience of using advanced analytics has been collected, a decision about scaling is the next step. A set of framing questions can be used to make an informed decision—these framing questions are adapted from Cooley and Linn7:
Scaling plan. Following best practices, an overall scaling plan should be developed, including scaling vision, scaling process, change management process, and supporting strategies. Our experience is that few organizations have an overall vision defined to become data-driven. Instead, data-driven is used as a tool to support more business-oriented visions. The scaling process should describe who is involved in the scaling process and what type of decisions and analytics are scaled first. Similarly to the initial work, a cross-functional group should be involved in the scaling process. Finally, a suitable change management process should be selected and adapted.
Strategies. In order to avoid the pitfall of mainly developing technical strategies for scaling advanced analytics, developed strategies should align with what it means to be a data-driven organization, for example, by using a sample framework for analytics.
By following the framework in the figure here, strategies developed include:
Although most of the strategies seem obvious, organizations tend to focus on developing mostly technical strategies or are still developing them.3 For example, 53% of the respondents in the 2022 survey of Fortune 1000 organizations had developed a corporate data strategy.13 To avoid working in silos, developed strategies for scaling advanced analytics need to consider and adapt to existing projects and strategies in, for example, AI, master data, or cloud computing.
The distinction between AI and advanced analytics is sometimes blurred as they can rely on the same tools and techniques.
Monitor and evaluate. Progress toward the vision at the organizational level is easily detected, for example, new departments, new roles, new policies, or new staff. Assessing what happens in practice at the team level requires a deeper analysis, preferably using analytics to assess what characterizes teams that have established a data-driven culture.
The distinction between AI and advanced analytics is sometimes blurred as they can rely on the same tools and techniques. According to Cam and Chui5 and Davenport and Malone9 most AI and advanced analytics models, for example, machine learning models, are not released into production. However, organizations that are top performers in deploying AI and analytics models into production have eased the deployment by Bisson et al.,4 Cam and Chui,5 and Davenport and Malone9: involving stakeholders early, cross-functional teams, focus on data governance strategies, new roles (for example, product managers, translators, AI strategist) that bridges data scientists with business users, aligning with important corporate strategies, and continuous learning programs (for example, in data literacy).
The key elements for scaling advanced analytics presented here align well with what top performers in deploying AI and analytics models focus on and best practices from change management. What is unique about scaling advanced analytics? If we put aside proper use of change management and having data strategies in place, our answer would be: an increased focus on establishing new roles that bridge technical experts and business users; and reskilling employees in data literacy.
12. Hart, D. and Warne, L. Comparing cultural and political perspectives of data, information, and knowledge sharing in organisations. International J. of Knowledge Management (IJKM) 2, 2 (Apr. 2006), 1–15.
We are very grateful for the comments received by the anonymous reviewers. The research is partially supported by VINNOVA, Sweden's innovation agency.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2023 ACM, Inc.
No entries found