Performance measurement and program evaluation are two ways juvenile justice program managers and staff members can assess what a program is trying to accomplish, how it is functioning, and what results are being achieved.

Performance measurement is a means to ensure fidelity in implementation.  Fidelity means program activities are being carried out as planned.  While performance measurement tells us a lot about a program, evaluation is needed to show that the program’s activities caused any observed outcomes.

Previous  Next

Dear Ms. Austen,

We regret to inform you that the Northwest Community Center is at risk for closure due to its failure to demonstrate the provision of effective, structured programming for NW Devonville youth. Due to the dramatic change in juvenile crime in Devonville, the state government has established new programmatic guidelines and evaluation requirements for all programs. In order for the center to be awarded funds for the next grant period, the city requires that you demonstrate the Northwest Community Center is using grant funds for an evidence-based program. Please note in the current program application that we are requiring the following evaluation-related components to be included for consideration for future funding:

1. Substantial information demonstrating the existence of the problem your program aims to address in the Northwest Community Center.
2. A detailed work plan of the proposed evidence-based program in your center, including a logic model that provides program goals, objectives, and activities.
3. The performance measures that will be used to track progress toward addressing the problem.

The Northwest Community Center has had a positive impact on various children’s issues in the past, but due to the increasing juvenile crime in the area, we can only support programs that clearly demonstrate a need and an evidence-based solution to the problem. We look forward to receiving your application for the next grant period. If you have any questions, please do not hesitate to call our office.

Sincerely,

Celia Hawkins

Celia Hawkins, Assistant Director
Office of Child Welfare and Social Services

Previous  Next

Darcy has a lot of work to do.  First, she needs to demonstrate the magnitude and nature of the problem by providing data.

To adequately define the problem, a program manager like Darcy needs to get a broad picture of all the factors contributing to the current situation.  This is best done by stating the facts: Who? What? Where? When? Why?

Darcy comes up with the following list of individuals she can contact to find out more:

• Local elected officials
• Community leaders
• Those working with juveniles in the community
• Concerned parents and teachers
• Media
• Law enforcement

Previous  Next

As stated earlier, residents in the NW quadrant of Devonville have been complaining about unsupervised juveniles hanging out in the neighborhood and committing crime during the day. Darcy needs to provide solid evidence of an actual problem to support their complaints.

Data are documented information or evidence of any kind.  There are a variety of sources where Darcy can obtain more information about her community to determine the extent of the juvenile crime problem, including:

• Police records
• School attendance logs
• Conversations with members of the community
• Census information
• State criminal and juvenile statistics agencies

Here are some suggested resources for assessing a problem in your state or local community:

• Uniform Crime Reports (UCRs)
• Census
• Statistical Analysis Center (SAC) websites

Previous  Next

In addition to justifying a problem in the community, program managers should also be able to identify characteristics of the juveniles they are targeting for particular interventions– the program’s target population.  Because the effectiveness of many interventions depends on how suitable they are for the population being served, youth characteristics such as age, gender, language, and problem behaviors should be considered, in addition to characteristics of the youths’ schools, families, and communities.  This can be accomplished through quantifying and qualifying the problem.

Here are some questions to ask in order to best define the problem you are seeking to address:

• Whom does it affect?
• How serious is it?
• What specifically is the problem?
• How does it connect with other problems in the community?
• What services are already in place to address the problem?
• What are the gaps in services?
• Why is there a problem?

Previous  Next

Darcy makes a trip down to the Devonville Police Department and meets with a crime analyst who pulls juvenile arrest statistics from their database. Darcy and and the crime analyst note that a large number of juvenile arrests in the NW quadrant are taking place during school hours.  Also, there has been a 5% increase in juvenile arrests in the NW quadrant in the past two years, while there has been little change in the juvenile arrests in the rest of the city.

Darcy also speaks with the principals in charge of the three public high schools in the NW quadrant to see if they have any helpful information.  Darcy learns that these schools have had a 25% increase in truancy and 20% increase in dropout among students in the last two years.  The principals note that these truancy and dropout increases are much higher than schools in other parts of the city of Devonville.

Based on the information from the police and school system, Darcy defines the problem as follows:

The NW quadrant has experienced a 5% increase in juvenile crime in the past two years. There has been no significant increase in juvenile crime in the rest of the city. In addition, there has been a 25% increase in the truancy and a 20% increase in school dropout for the public high schools in the NW quadrant for each of the last two years.  The increases in truancy and dropout are also unique to the NW area of Devonville, and may be contributing to the juvenile crime problem.

Darcy has decided to review research on programs that have addressed these problems effectively with youth similar to those in NW Devonville.  After reviewing this research, she will select an evidence-based program that is appropriate for her target population.

Previous  Next

The best question to ask when implementing evidence-based programming is: What is the linkage between the identified problem and the program you want to implement?  What evidence-based programs, practices, policies, or principles have addressed this type of problem successfully?

Darcy must be able to demonstrate:

1. Sufficient research exists to support the program plan as a possible solution for the problem Darcy wants to address.
2. The program is well established as being effective for this target population.
3. Results have been published and they are widely accepted.

Previous  Next

Not all programs and practices have the same amount of evidence.  Here are some frequently used terms to describe the level of support that exists for an evidence-based program or practice:

• Exemplary/model/best—clear evidence of effectiveness with multiple, rigorous evaluations;
• Promising—some evidence of success, many questions remain; or
• Innovative—derived from evidence-based programs, but has not yet been evaluated.

When Should I Use An Innovative Program?

An exact version of the program you envision does not have to exist in order for you to rely on evidence-based programming. Search the literature for programs which address issues similar to the problem you have identified. You may be able use an innovative program that is based in research evidence, but has not yet been sufficiently evaluated to be considered an evidence-based program.  You may also be able to combine strategies from more than one program.

Previous  Next

Darcy’s center is already well-attended by youth in the area, but she needs to focus the center’s activities to address the problems of truancy and dropout that seem to be contributing to the juvenile crime problem.

From her search of online resources, Darcy learns that increased truancy and school dropout rates are risk factors for juvenile delinquency.  The data she has collected indicate these factors may contribute to the current juvenile crime problem in NW Devonville.  Darcy’s research indicates that the center can most effectively address the juvenile crime problem with the strategies listed below.

Given the problem with youths skipping or dropping out of school, Darcy needs to find a program to encourage high school youth to attend school.

Since most arrests involve groups of juveniles, Darcy assumes that some juveniles are being negatively influenced by the antisocial attitudes of their peers. She will therefore want to identify a program that will boost juveniles’ prosocial values, including valuing social institutions such as schools.

Previous  Next

Darcy’s next step is to find programs that make use of these strategies to address delinquency.

Darcy compiles available information on evidence-based programs that target juvenile delinquency and are well-suited for high school youth in NW Devonville. She uses online resources to find programs that have demonstrated successes in multiple studies with strong evaluation designs.

After reviewing several programs, Darcy chooses a community-based mentoring program. One of the main reasons she chose this program is that it seeks to increase juveniles’ prosocial values, such as the importance of education, while at the same time directly addressing issues related to truancy and drop out, such as academic performance and participation in extracurricular activities.

Previous  Next