Understanding AI-enabled Outputs: A Conceptual and Analytical Primer for All-source Intelligence Analysts

Abstract

Intelligence analysts are increasingly able, and required, to consume and interpret outputs produced by artificial intelligence (AI) enabled tools – yet most receive little training in what these outputs actually represent, or how these might be robustly evaluated. This primer addresses that gap. It starts from the premise that analysts do not need to know how to develop or operate AI-enabled tools in order to use these tools’ outputs critically and judiciously. But analysts do need sufficient conceptual and analytical understanding – and a modicum of technical knowledge – to evaluate these outputs competently. Three principles provide the framework for this understanding: First, the consequential distinction between AI-facilitated outputs – where automation improves the pace, scale and fidelity of data collection, processing and analytical procedures that analysts could otherwise perform; and AI-generated outputs – where many of the novel insights these outputs support could not have been produced by analysts working independently; Second, the conceptually challenging yet fundamental difference between mechanistic prediction and interpolative or extrapolative estimation; and Third, the critical dependencies and substantive limitations that govern the reproducibility and practical utility of all AI-facilitated and AI-generated outputs. Together these principles constitute the conceptual and analytical foundations of the AI literacy training that all-source intelligence analysts should receive – the case for which is presented in a companion piece to this position paper.