Planning Data Investigations


Leon Yin


August 13, 2023

Data investigations can get complicated and you might not know where to start.

In this section we’ll introduce you to a planning checklist that we use for our own investigations.

The checklist will help you choose an accountability angle, identify tangible harms, and form a testable hypothesis.

Most importantly, the checklist covers questions you’ll need to answer in order to develop a defensible methodology and a bullet-proof story.

Your readers and future self will thank you for being forthright about your investigation’s vulnerabilities. Moreover, you’ll get a sense of the experiment’s feasibility before you invest too much time into it.

Although grim, the checklist can help expedite the decision to kill a story. Killing a story is a difficult process– there’s even a podcast on the topic called Killed Stories, but it’s better to catch fatal flaws and irreconcilable uncertainties early. Trust that doing this efficiently is a gift to yourself and your colleagues.

The questions in the checklist cover fundamental topics we’ll discuss throughout the practitioner’s guide such as:

  1. Data collection
  2. Viability (minimal viable) analyses
  3. Classification
  4. Statistical tests
  5. Limitations
  6. Communicating findings for a general audience

You won’t need to fill out a checklist for every story, but it is super helpful for projects with original data collection and/or analysis.

Lastly, view the checklist is a starting point. Add and edit questions with your team to assure you can publish your findings with certainty.

Good luck.