The Problem Is Not Data. The Problem Is Your Brain.
In March 2009, the US Government published a slim, unclassified volume titled A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis. It was written for people who track missile programs and terrorist cells, but its central diagnosis applies with equal force to a detective sergeant staring at a whiteboard covered in string. The primer’s thesis, borrowed from Richards Heuer’s The Psychology of Intelligence Analysis, is blunt: analysts do not fail for lack of information. They fail because human beings process information through “mental models” that decide, before conscious thought kicks in, which facts get noticed and which get quietly binned. Analysts “perceive what they expect to perceive,” the primer warns; once a mind-set forms it “resist[s] change,” and “conflicting information is often dismissed or ignored.”
Anyone who has worked a homicide unit will recognize that description as a clinical portrait of tunnel vision. The wrongful conviction literature is a graveyard of cases where investigators locked onto a suspect in the first seventy-two hours and then spent months unconsciously bending every new fact to fit. The primer’s most damning observation is that experience makes this worse, not better. “Seasoned analysts may be more susceptible to these mind-set problems as a result of their expertise and past success.”
What follows is my argument that the primer’s twelve techniques, grouped as diagnostic, contrarian, and imaginative, constitute a ready-made discipline for law enforcement crime analysis, and that departments which ignore them are choosing to run investigations with the analytic rigor of a bar argument. The primer already makes this case implicitly. Its showcase example for the Key Assumptions Check is not a foreign intelligence problem at all. It is the 2002 Washington, DC sniper investigation.
Diagnostic Techniques: Making the Argument Visible
Key Assumptions Check
The primer defines a key assumption as “any hypothesis that analysts have accepted to be true and which forms the basis of the assessment.” The method is four steps: write down the current analytic line, articulate every stated and unstated premise beneath it, challenge each one, then keep only those that “must be true” for the line to hold.
The DC sniper case shows why this matters. The operating theory that “quickly emerged” held that the shooter was a lone white male with military training driving a white van. Broken into components and scored honestly, the primer notes, only “male” and “acting alone” rated as highly likely on precedent. “White” was merely likely. “Military training” was possible but “not sufficient reason to exclude” untrained suspects. And the white van rested on a single eyewitness in a region with more than 70,000 registered white vans. Officials who had done this exercise “could have avoided prematurely narrowing down the potential pool of suspects to a group that did not include the actual perpetrator”—two Black men, one a teenager, in a blue Chevrolet Caprice. The tip about that Caprice came in. It got lost in the noise because nobody had written down that they were assuming a van.
The application to routine crime analysis is direct. A serial burglary pattern assumes the offender is local, works alone, and is fencing goods rather than hoarding them. A gang shooting assumes retaliation for the last gang shooting. A missing-person case assumes the spouse. These assumptions are usually right, which is exactly why they are dangerous: nobody writes them down, so nobody abandons them when the evidence quietly stops supporting them. The primer’s discipline is not to reject working assumptions but to make them explicit and, crucially, to “identify what information or developments would demand rethinking them.” A crime analyst should hand every case team a one-page assumptions register before the first canvass. It costs an hour. It has saved careers.
Quality of Information Check
Intelligence agencies learned the hard way, in the run-up to Iraq, that “over-reliance on a single, ambiguous source” and failure to understand “the limitations of technical collection” can wreck an entire analytic enterprise. Police departments learn the same lesson, one jailhouse informant at a time.
The primer’s method is a periodic, systematic review of every source feeding an analytic line, with attention to “the context and conditions under which critical information has been provided.” For human sources, that means “extensive review of the sources’ background information and access as well as his or her motivation.” Translate that into policing and you have a checklist that would embarrass a great many affidavits: What does the confidential informant get for this? Has this CI ever been wrong? Was the eyewitness identification made under conditions that the psychological literature says produce reliable results, or was it a six-pack shown by the lead detective who already had a favorite?
The primer adds a point that should make every prosecutor sit up: checks must ensure “that any recalled reporting is identified and properly flagged,” and that “analysis based on recalled reporting” is re-examined to see whether it “was essential to the judgments made.” In the criminal justice context this is the Brady/Giglio problem viewed from the analyst’s chair. If a source is discredited, the question is not merely whether the defense gets told. It is whether the entire investigative theory was load-bearing on that source, and whether anyone has the guts to say so out loud.
Indicators or Signposts of Change
Crime analysts already do indicator tracking; they just call it a hot-spot map. The primer’s contribution is to formalize the logic. You identify competing hypotheses or scenarios, build a separate list of observable events expected under each, review the lists on a schedule, and let the pattern of changed indicators tell you which scenario is materializing.
The primer’s illustration is a political-instability matrix that tracks government capacity, regime legitimacy, opposition activity, economic factors, and “trigger mechanisms” across quarters, color-coded from negligible to serious concern. Swap the rows and you have a gang-conflict early-warning matrix: recent releases from custody, social media taunting, shifts in narcotics pricing, unexplained gunshot detections in contested blocks, a funeral on the calendar. No single indicator predicts a shooting, but a disciplined, periodically reviewed list “depersonalize[s] the argument” between the gang unit sergeant who insists things are quiet and the analyst who thinks the neighborhood is about to catch fire. Once both sides agree on the criteria, the data does the arguing.
Analysis of Competing Hypotheses
ACH is the primer’s heavyweight, and it is the technique law enforcement most needs and least uses. The method: brainstorm every reasonable hypothesis, list every significant piece of evidence, build a matrix with hypotheses across the top and evidence down the side, and score each cell as consistent, inconsistent, or not applicable. Then comes the part that breaks most investigators’ instincts: “Focus on disproving hypotheses rather than proving one.”
The primer’s example scores four explanations for the 1995 Tokyo sarin attack (cult, terrorist group, political movement, criminal group—against four pieces of evidence, and notes that evidence consistent with every hypothesis (the group had a blind leader) has zero diagnostic value. A suspect with no alibi, a motive, and proximity to the scene sounds like a strong case. Run those three facts against the six other people who also had no alibi, a motive, and proximity, and they discriminate among no one. Detectives collect evidence that is consistent with their suspect. ACH forces them to ask which evidence is inconsistent with everyone else.
ACH also produces a defensible record. The primer notes it is “particularly appropriate for controversial issues when analysts want to develop a clear record that shows what theories they have considered.” A cold-case unit or a conviction-integrity review that can show a matrix of considered and rejected hypotheses is in an infinitely stronger position, in court and in the press, than one that can only say “we always knew it was him.”
Contrarian Techniques: Institutionalized Disagreement
Devil’s Advocacy
When “a group of analysts have worked on an issue for a long period of time, it is probably wise to assume that a strong mind-set exists.” The primer’s remedy is to designate someone (it uses the word “courageous”—to build the best possible case for an alternative explanation, then present findings that expose “flawed assumptions, poor quality evidence, or possible deception.”
Every major-case unit should have a standing devil’s advocate who is not the lead investigator and does not report to the lead investigator. Before charging, this person’s job is to write the defense’s closing argument. If the case survives, it is stronger; if not, you just saved the county a seven-figure settlement. The primer’s warning about labeling applies with special force: a devil’s-advocate memo in a case file must be clearly marked as such, or a defense attorney in discovery will wave it around as the department’s “real” view.
Team A/Team B
Where Devil’s Advocacy challenges one consensus, Team A/Team B stages a structured contest between two genuinely competing views. The primer recommends it for “a dispute within the analytic community that has obstructed effective cross-agency cooperation,” which describes roughly every joint task force ever assembled. The narcotics squad believes the overdose spike is a cartel supply shift; the homicide unit believes it is a local crew cutting product with fentanyl to stretch margins. Each team builds its case with explicit assumptions and evidence, presents to a “jury of peers,” and takes questions. The primer’s practical tip is superb: put analysts on the team arguing the position they don’t hold. Nothing exposes your own mind-set faster than being forced to argue against it.
High-Impact/Low-Probability Analysis
The primer opens this section with a State Department memo dated 27 November 1941 offering five-to-one odds against war with Japan within three weeks. Pearl Harbor was ten days later. The technique is designed for exactly this failure. That events “analysts and policymakers will be convinced” is unlikely but “have not given much thought to the consequences of” is extremely likely.
For a police department, the list writes itself, a coordinated ambush of officers, a mass-casualty attack at the county fair, a courthouse shooting, a targeted killing of a judge or prosecutor, . . . the list goes on. The method asks the analyst to define the outcome, devise plausible pathways to it, insert triggers, generate indicators for each pathway, and identify factors that would deflect it. It is the difference between thinking about the unthinkable on a Tuesday afternoon with coffee and thinking about it for the first time on the radio.
“What If?” Analysis
Closely related, “What If?” simply assumes the event has happened and works backward. The primer’s example is a 1990 National Intelligence Estimate on Yugoslavia that assessed muddling-through as very unlikely, then spent a page describing precisely what it would look like if it happened. The technique is valuable when “a judgment rests on limited information or unproven assumptions,” which is most judgments. Assume the confidential informant has been playing the squad for a year. Assume the suspect in custody is innocent. Now think backward, concretely, to what must have happened at each stage for that to be true, and list the observables.
Imaginative Techniques: Getting Outside the Building, the “Box” or Whatever the Hell You want to Call It.
Brainstorming
The primer’s brainstorming is not a bull session; it is a “very structured process,” with a divergent phase (silent idea generation on sticky notes, no censoring, at least one “outsider” present) followed by a convergent phase (silent clustering, then voting). Its most transferable rule is leaving “rank at the door.” The practitioner quoted in the primer observes that the best ideas often come “from relatively junior people who can look at a problem with fresh perspective.” In a paramilitary organization where a lieutenant’s theory becomes the squad’s theory by default, this is a structural correction, not a nicety. Invite the patrol officer who works the beat. Invite the civilian analyst. On a cold case, invite a defense attorney and watch the room squirm.
Outside-In Thinking
Crime analysts spend their days “inside”: the calls for service, the arrest data, the known offenders. Outside-in thinking starts with the external forces—”social, technological, economic, environmental, and political”—over which the department has no control but which shape the problem anyway. Eviction filings, a plant closure, a change in state sentencing law, a new encrypted messaging app, a heat wave, the school calendar, an upstream shift in fentanyl precursor supply. The primer’s promise is modest and correct: a small effort at the front end “can reduce the risk of missing important variables early in the analytic process.”
Red Team Analysis
The primer warns against “mirror-imaging”: assuming an adversary thinks as you would. Red Team analysis puts analysts inside the target’s “cultural, organizational, and personal setting” and has them draft first-person products from that vantage. For crime analysis, this means assembling people who actually understand the operating environment—the primer insists on those who “might have experienced the culture”—and having them plan the crime. Where would I fence forty stolen catalytic converters? How would I move product if the corner I worked just got a camera? What would my crew expect me to do after my cousin got shot? The primer is candid that this is hard and that analysts “can never truly escape their own experiences.” But a department that has never once tried to think like the people it is chasing is not doing analysis; it is doing bookkeeping.
Alternative Futures Analysis
Finally, for the strategic planners: scenario analysis. Identify the two most critical and uncertain drivers, cross them into a 2×2 matrix, and write “colorful stories” describing each of the four resulting worlds. The primer’s homeland-threat example crosses weapon sophistication against intended breadth of impact. A department could cross narcotics supply (constrained/flooded) against enforcement funding (cut/sustained), or population growth against court capacity, and ask which current strategies survive in all four quadrants.
Sequencing, and a Warning
The primer closes with a timeline, i.e., brainstorming, assumptions checks, and outside-in thinking at the start; contrarian techniques and red teaming as hypotheses firm up; a final assumptions check and devil’s advocacy before the product ships; ACH and indicators running throughout. Map that onto a major case (initial response, suspect development, pre-charge review) and the fit is nearly exact. However, I have some cautions. First, the primer is honest that these techniques are “no guarantee of analytic precision.” They improve the “sophistication and credibility” of judgments; they do not manufacture evidence, and an ACH matrix is not probable cause. Second, in law enforcement these products are discoverable. That is a feature. An investigation that can show its assumptions, its considered alternatives, and its source-quality review is an investigation that survives cross-examination. The one that cannot is relying on the jury never asking how the sausage got made.
Intelligence analysts adopted this discipline because they got burned, publicly and catastrophically, by their own mind-sets. Policing has the same failure mode and, in wrongful convictions and unsolved cases, the same body count. The framework is public, it is free, and it is thirty-nine pages long. There is no excuse left for not using it. Intelligence, counterintelligence and their corresponding directorates of analysis invented the wheel. I write about them frequently and put them in the wild in hopes that fellow I.C. and LEO analytical cadres don’t reinvent that wheel.