What the CIA Can Teach us About Business Intelligence

Amid all the generalizations and prescriptions about how executives, their analysts and others in a company should produce and consume reports about their company’s metrics/data, I was naturally curious to find a more scientific take on the matter. To shed some light on this matter, I explored research done by Peter Pirolli and Stuart Card. They have studied information gathering and consumption in the high stake setting of national security and intelligence activities. Their insights can effectively be applied to more traditional business settings.

Pirolli and Card model the information lifecycle as beginning with the gathering stage, then the representation of that information in a schema and finally the manipulation of that schema (looking at it from different angles, timeframes, etc.) to find insight. It sounds deceptively straightforward. But they have spent a lot of time in the field watching these processes happen in real time and have created a very interesting model, down to quite a granular level.

The image above depicts two information loops. The first is a foraging loop whereby the analyst extracts the information and filters it. The second loop is the sensemaking loop wherein the analyst builds a mental model of the evidence and a story around it.

[As an aside, most business intelligence tools attempt to cover both loops. Where they typically fail is on the second, sensemaking loop. This failure is typically one of poor user experience and an overly complex and thus unusable product].

The really interesting part of this loop is the implicit tradeoff curve of time spent foraging for interesting and novel information versus speed in producing actionable or useful reports. Speaking abstractly, in order to find more novel or anomalous (and thus interesting data), the diligent analyst needs to lower their filters, so to speak, and let in more irrelevant information (i.e. noise). This comes at a cost to their speed and potentially leads to noisy thinking.

In a passage particularly relevant to Business Intelligence tools, the authors claim:

“One class of solutions (that comes by analogy to human sensory and perceptual systems such as vision) would be to produce human-information interaction systems that foster broad-band, low-fidelity assessments of incoming data coupled with narrow-band, high-fidelity processing. Focus+context techniques (Furnas, 1986; Furnas, 1983) are examples of these kinds of solutions.”

This hits the mark exactly. The best BI tools out there allow this kind of focus/context split through drill-downs that echo the way we humans think intuitively. What we do here at Chartio and our colleagues in the Business Intelligence, Analytics and Data Visualization space do every day is attempt to convert information into sense, or meaning, whether it be through an innovative interface that lets users easily gain context and depth with drill downs or a new type of data visualization. Stuart Card made a brilliant point about this process of sensemaking, which we just explored in the context of intelligence analysts. 

“Looking back on the tenth anniversary of the first Symposium on Information Visualization, much has been accomplished. The computer’s power has been exploited to give quick visual form to abstract data, to interact, and to warp detail to follow the user’s changing interest. Moreover, the design space of visualizations has been systematized with reference models, taxonomies, and monographs relating visualization to perceptual and graphical constraints. Looking ahead, however, I will argue that the era of pure information visualization is over. The path ahead depends on giving much more attention to the purposes of visualization and its use. Leaving aside communication, the purpose of information visualization is insight, or more particularly, a larger process that might be called sensemaking. I will sketch out the nature of sensemaking, exemplify it empirically in a practical, urgent setting, and suggest how theories of sensemaking could be developed. I will then describe systems that subsume information visualization as part of an emerging class of sensemaking systems combining visualization (the mind’s eye) with semantic content analysis and sensemaking operations (the mind’s muscle). Not surprisingly, a focus on sensemaking is a good generator of new visualizations. But these developments also suggest that it may be time for the information visualization field to alter its boundaries to go beyond the merely visual.”

We couldn’t agree more.