You might be tempted to show that a program is working by providing anecdotal evidence. An individual might cite one or two instances of a certain result as “proof” that a program is working effectively without considering whether the results are representative, or if the results were actually caused by the program instead of other factors.
For example, suppose a mother calls Darcy three months into her mentoring program to report that her son’s behavior at home is ten times better than it used to be. Her son is now taking out the trash without having to be asked; he’s using the phone much less; he’s obeying his curfew. These are signs that something is changing the youth’s life, but these changes may not be representative of the program outcomes. Other youth in the program may not be showing the same improvements, or this youth’s behavior might be changing for some reason unrelated to his participation in the mentoring program. Darcy cannot assume the program is a success overall based on the results experienced by this parent.
Systematic data collection—collecting the same data from all individuals participating in the program—gives better information about the impact of the program on its target group.