“Flexibility is important, but it has to do with the mindset that people have. There’s a mindset where analytics is telling me what the right answer is, and then there’s the mindset that analytics is a tool to help me analyze the situation.”
The popular movie, Moneyball, follows the experiences of the Oakland Athletics baseball team’s 2002 season, during which the team’s General Manager, Billy Beane (played by Brad Pitt), attempts to rebuild a losing team. Beane and his Assistant General Manager, Peter Brand (played by Jonah Hill), face a difficult conundrum—rebuilding a losing franchise with a limited budget.
Beane and Brand implement a radical strategy: instead of relying on the scouts’ experience and “gut-feelings” about players, they assess player abilities using statistical metrics like on-base percentage (OBP). The Oakland A’s managers create a successful team using sabermetrics—analytics applied to baseball. In the movie (and in real life), the team goes on to achieve an American League record by winning 20 consecutive games.
Moneyball highlights a tension between “objective,” “evidence-based” decision-making based on analytics and “subjective,” “biased” decision-making based on the opinions of human experts. As in the movie, many companies view insights produced from analytics and insights produced from human experts as competing sources of knowledge. In other words, many people believe that organizations primarily rely on analytics to make sure employees make decisions in an objective, unbiased manner.
This notion that analytics automates expert decision-making is too simplistic: effective implementation of analytics occurs when an organization uses analytics to augment and enhance the hard-earned expertise of its experienced team members. It’s analytics and experience, not analytics or experience.
This leads to a simple question: How can a public safety organization adopt analytics effectively without undermining the valuable experience of its team?
Defining key performance metrics
The first step in adopting analytics effectively requires organizations to build on a simple—but necessary—foundation: key performance metrics that summarize routine activities. Examples of routine activities in public safety organizations include fire trucks or ambulances responding to 911 calls and police officers conducting patrols to prevent crime. These metrics must accurately measure the performance of the organization, and your team has to believe in them.
Chances are, your organization already has a number of metrics in place. For example, an emergency services organization might evaluate performance of its 911 response routine by measuring the time it takes for a unit to respond to a call; a law enforcement agency might evaluate performance of its officers by measuring the number of cases cleared.
Often, organizations use these metrics simply as a means of accountability: team members who consistently miss performance targets are assumed to exert less effort than team members who meet or exceed targets. However, analytics can help an organization go beyond such simple attributions. It helps reveal the root cause of variations in performance.
Using analytics to understand variations in performance
Discovering the root cause of variations in performance is not easy. Although popular media might make people think that “machine learning” can replace human experts, the reality is that the most effective way for a public safety organization to use analytics is for root cause analysis. Analytics provides the tools they need to sift through mountains of data and figure out where to focus their attention.
Specifically, seasoned team members often have theories about the causes of performance issues, but struggle to gather evidence to test those theories. To develop initial theories, team members can ask some basic questions:
- Who performed the routine?
- Where did they perform the routine?
- What equipment was used to perform the routine?
- What tasks were performed in the routine?
- When was the routine performed?
Often, just looking at the answers to these questions will point you in the right direction. Ideally, analytics helps public safety team members rapidly test their theories and narrow in on the most fruitful interventions. But beyond just identifying problems, it also helps you quantify which sources of variation in routine performance will have the most positive impact on the key performance measurement. Analytics saves you time and reduces wasted effort.
Conclusion
The bottom line: using analytics to understand performance does not replace the judgment of seasoned professionals. Rather, analytics can help the experienced leaders of an organization develop and test hypotheses about how they can improve performance. Put another way, analytics is not just about having an expert come in and do everything for you; analytics is about using the right tools and resources to help your organization think more analytically and understand how you can more efficiently meet your objectives.
About Vern Glazer
Vern is an Assistant Professor in the Strategic Management and Organization Department at the University of Alberta. His current research revolves around four inter-related concepts naturally associated with organizational efforts to change practices and/or culture: language, tools and technology, professional expertise, and organizational consciousness. Vern is currently doing research on analytics adoption with the Strathcona County Emergency Services. Visit Vern’s website to learn more about his work.
