Creating a Logic Model

Welcome to Module Three!

There are seven key steps in the evaluation and performance measurement process.  In Module 3, we will focus on Steps Three and Four:

  1. Define the problem.
  2. Implement evidence-based programming.
  3. Develop program logic.
  4. Identify measures.
  5. Collect and analyze data.
  6. Report findings.
  7. Reassess program logic.

Previous  Next

What is a Logic Model?

Darcy has identified a problem in her community and selected an evidence-based mentoring program to implement.  Darcy’s next step is to define the elements of the program.  This visual depiction of a program plan is called a logic model.

While there are several ways to set up a logic model, typical components include:

  • GOAL(S)
  • OBJECTIVES
  • RESOURCES
  • ACTIVITIES
  • PROCESS MEASURES
  • OUTCOME MEASURES
  • EXTERNAL FACTOR

Click here for a blank logic model template you can print and fill in as Darcy develops her logic model.  Remember, you can set up your own logic model in the way that best presents your program’s main components– this is just a sample.

 

Previous  Next

What is a Logic Model?

A logic model provides:

  • A format for identifying what the program expects to achieve.
  • A basis for monitoring activities.
  • A method to document what the program intends to do and what it is actually doing.

 

The logic model helps maintain focus and program fidelity by linking project goals, objectives, activities, and performance measures. It clearly shows which activities should be carried out and for what purpose, as well as what data should be collected to keep track of program activities.

Darcy can include her logic model in her grant application to explain how her program will address rising juvenile crime in NW Devonville, and how she will demonstrate that the program is following this plan.

Previous  Next

Step Three: Develop a Logic Model

To begin creating a logic model, Darcy needs to develop a GOAL statement for the mentoring program.

A goal is a broad statement about what a program intends to accomplish.  A goal addresses the intended long-term outcome of a program.

 

Darcy comes up with three possible goal statements based on the above definition:

  • To hire 50 mentors within two months for the program.
  • To rid the Devonville community of all juvenile crime.
  • To prevent juvenile delinquency in NW Devonville by strengthening high school students’ bonds to their school.

 

Based on what you know about Devonville, which goal statement do you think is most appropriate for Darcy’s mentoring program? Remember, it should be broad and describe the long-term changes the mentoring program expects to achieve in Devonville.

Previous  Next

Step Three: Develop a Logic Model

Here are some possible goals for Darcy’s program:

 

To hire 50 mentors for the program within two months.

The goal must be broad and tell what the program expects to achieve in terms of the problem. To hire 50 mentors doesn’t say anything about what the program is expected to accomplish. Also, it is not a statement of what the program expects to achieve, it is a statement about a program activity.

 

To rid the Devonville community of all juvenile crime.

This goal statement is definitely broad, but it is unrealistic.

 

To prevent juvenile delinquency in NW Devonville by strengthening high school students’ bonds to their school.

This goal is broad, offers a solution to the crime problems in Devonville, and is realistic, assuming that everything Darcy has in mind for her program goes as planned. Great! Darcy has the program’s goal!

Previous  Next

Step Three: Develop a Logic Model

Each GOAL has one or more OBJECTIVES associated with it.
Each OBJECTIVE has a set of ACTIVITIES with which it is associated.

If the activities are carried out successfully, they will lead to the accomplishment of the program’s objectives, which will ultimately lead to the achievement of or progress towards the goal.

The set of relationships between a program’s goals, objectives, and activities are often referred to as program theory or program logic. These components specify the process by which the program will produce positive changes in the youth it serves.

Darcy uses the information she has collected about the evidence-based mentoring program to help her develop her program logic.

 

Previous  Next

Step Three: Develop a Logic Model

Darcy’s program theory (also called program logic) should explain how the mentoring program will benefit Devonville and its youth. The logical relationships between the program’s goal, objectives, and activities can be expressed through a series of IF-THEN statements:

if then graphic

IF high school youth develop positive relationships with mentors who help with homework and studying, monitor attendance at school, and encourage and facilitate participation in extracurricular activities,
THEN youth’s grades will improve, attendance will improve, and participation in extracurricular activities will increase.

IF high school youth’s grades improve, attendance improves, and participation in extracurricular activities increase,
THEN youths will feel stronger bonds or ties to their schools.

IF high school youth feel strong bonds or ties to their schools,
THEN they are more likely to adopt other prosocial values such as a belief in the importance of lawful behavior.

IF high school youth adopt other prosocial values,
THEN they are less likely to engage in delinquent behavior.

Previous  Next

Step Three: Develop a Logic Model

This is Darcy’s theory of how a mentoring program will help the students of Devonville.  Her program logic spells out how the goal, objectives, and activities relate to each other.

activities graphic
 

Establishing these associations allows Darcy and any evaluator she may work with to examine each individual connection, and to see where the connections succeed or fail. She can continually revisit the logic model to be sure her program is following its original plan. If adjustments are needed to improve the program or data collection, the program logic and logic model can be updated.

Previous  Next

Step Three: Develop a Logic Model

In addition to a logic model, Darcy should also provide the city with a program narrative. A program narrative includes a detailed description of the mentoring program, including how the program is organized, who the target population is, and where the program is located.

Putting ALL of this information on a logic model would cause it to be too cluttered. The logic model should be easy to read and understand very quickly, and the narrative provides all the extra (but important!) details about the program.

Darcy comes up with the following description. Note that she included a number of important features of the program, including the program plan’s evidence base, target population, program activities, location, and time needed to carry out the program plan.

Previous  Next

Program Narrative

The Northwest Community Center Mentoring Program

The Northwest Community Center Mentoring program is for Devonville youth who are truant or may drop out of school.  This program is based on an evidence-based mentoring program that has repeatedly demonstrated effectiveness across a range of settings. The program targets a number of the behavioral problems related to juvenile delinquency in NW Devonville, such as academic performance and attitudes towards school.

The program will be hosted at the Northwest Community Center. In order to be eligible for the program, youth must meet the following criteria:  Live in the Northwest area of the city; be a high school student between the ages of 14 and 18; have a GPA for the first quarter of school below a C average; have accrued three or more unexcused absences during the first quarter of school year; and agree to be mentored for at least one year.

Youth are referred to the program by a parent or guardian, a teacher, and/or by a school guidance counselor. Upon referral to the program, the program manager, Darcy Austen will schedule a meeting with the identified youth, his or her parent(s) or guardian (s), and at least one teacher within two weeks to develop a plan outlining all of the activities that the youth must partake in throughout the duration of the program. The plan is subject to change based on the youth’s progress.

Additionally, the youth will be matched with an adult mentor who will contact him or her within two weeks of the first meeting and the signing of the program consent form (which is considered the youth’s start date in the program).  For the duration of the program, mentors will meet with their assigned youth two times per week to go over homework and/or other academic issues.  All youth and their mentors will participate in a community-sponsored workshop on the requirements for being admitted to college.

For the duration of the program, mentors will contact their youths’ teachers weekly to discuss and address any problems related to truancy and/or any other academic issues. Mentors and their assigned youth will meet at least four additional times throughout the duration of the program to discuss youths’ interest and involvement in after school activities (sports, clubs).  Throughout the program, all youth will be assessed based on their academic performance, attendance in school, and effort they put forth with their mentors.

Previous  Next

Step Three: Develop a Logic Model

Now that Darcy has established her goal and written her narrative, she turns her focus to developing other components of the logic model:

OBJECTIVES:  Expected achievements that are well defined, specific, measurable, and derived from the goal.

ACTIVITIES:  The steps or tasks undertaken to meet the objectives of the program.

Darcy should also keep in mind the resources that are available to operate the program and to accomplish the program’s goals and objectives. Below are some examples of resources:

  • Building to house a program
  • Staff to manage and direct the program activities
  • Computers and staff to collect data and conduct analyses
  • Funding to support program objectives

There may be external factors such as the availability of afterschool activities that affect Darcy’s program.  If they seem significant, she might want to include these external factors in her logic model.

Previous  Next

Step Three: Develop a Logic Model

Here are possible objectives, resources, and activities that could be associated with the goal of Darcy’s program:  To prevent juvenile delinquency in NW Devonville by strengthening students’ bonds to their school.

Objective #1: Develop positive relationships between NW Devonville youth and their mentors by completion of the program.

 This is a good objective because it describes an initial expected effect of the program (developing relationships), it includes a direction (positive), a time frame (by program completion), and a target population (youth in NW Devonville).  Keep in mind that this is only one of several possible objectives for the program.

Activities: Each youth participating in the program will be matched with one adult mentor within two weeks of beginning the program.

To determine if this activity is appropriate, consider whether it appears to be a task that would contribute to meeting the objective.

Try to come up with your own objectives and activities for Darcy’s mentoring program. Click to the next slide for another objective example.

Previous  Next

Step Three: Develop a Logic Model

Objective #2: Increase the grade point average for youth in Northwest Devonville participating in the program within 6 months of being matched to their mentor.

Activities: For the duration of the program, mentors will meet with their assigned youth two times per week to go over homework and/or other academic issues.

 

Let’s think about some of the resources needed to carry out program activities.

Resources: Access to school records, text books, communication between school and mentor.

Previous  Next

Step Four: Identify Measures

Now that Darcy has defined the problem, selected an evidence-based program, and developed the program logic, she can proceed to the next step in the evaluation process: identifying performance measures.  These measures should assess progress toward reaching the goals and objectives.

There are two types of performance measures: process measures and outcome measures.

PERFORMANCE MEASURES

PROCESS MEASURES: Data used to demonstrate the implementation of activities. These include products of activities and indicators of services provided.

OUTCOME MEASURES: Data used to measure achievement of objectives and goal(s).

 

Previous  Next

Step Four: Identify Measures

PROCESS MEASURES

Process measures are able to tell us whether the program is being implemented according to the original plan.  If substantial changes are made to an evidence-based program or practice, it may not work as effectively, so it is important to collect process measures on the execution of program activities.  Fidelity in implementation means that the program’s activities adhere to the program plan.

Darcy should collect at least one process measure for each activity, so that all program activities during the grant period are tracked. It is important to accurately track all activities to ensure the program was properly implemented.  These measures may also be used to determine why program results differ from what was expected, should that occur.

 

Previous  Next

Step Four: Identify Measures

OUTCOME MEASURES

Outcome measures show the change (or lack of change) in the target population that are directly related to the goal(s) and objectives.

There are three types of outcomes: initial, intermediate, and long-term.

  • Initial outcomes are the immediate results of the program.
  • Intermediate outcomes are the results following the initial outcomes.
  • Long-term outcomes refer to the ultimate impact of the program and are related to the achievement of the program goal.

Note that a program may not have each of these types of outcome measures, depending on program design and ability to collect the data required for the measure.

To develop outcome measures, think about what information the program will collect to document what you expect to occur as a result of the activities.  What difference do you expect your program to make, for whom, and by when?

Previous  Next

Step Four: Identify Measures

All performance measures should be:

  • Well-defined
  • Specific
  • Measurable
  • Derived from the goal

Let’s begin by choosing process measures for Darcy’s mentoring program.

Previous  Next

Step Four: Identify Measures

The main question to ask when thinking about process measures is this: What information should I collect to document what the program is doing?  Here are some of the ACTIVITIES Darcy would like to document:

  1. Each youth participating in the program will be matched with one adult mentor within two weeks of beginning the program.
  2. For the duration of the program, mentors will meet with their assigned youth two times per week to go over homework and/or other academic issues, as well as discuss youths’ interest and involvement in after school activities (sports, clubs).
  3. All youth and their mentors will participate in a community sponsored workshop on the requirements for being admitted to college.
  4. For the duration of the program, mentors will contact their youths’ teachers weekly to discuss any problems related to truancy and/or any other academic issues.
  5. If problems are identified, the mentor will address issues with youth.

 

Darcy proposes these PROCESS MEASURES to assess the program’s activities:

  1. Number and percent of youth in the program matched with an adult mentor within two weeks of the beginning of the program.
  2. Number of meetings per week attended compared to total number of meetings possible (twice a week for the duration of the program).
  3. Number and percent of youth/mentor pairs who attended the college admissions requirements workshop compared to the number of youths participating in the program.
  4. Number of contacts made with youths’ teachers compared to total number of weeks in program.
  5. Number and percent of problems identified by teachers that were addressed in youth-mentor meetings.

Previous  Next

Step Four: Identify Measures

Darcy selects her outcome measures by determining the change(s) she assumes will result from program activities.  Here are some of Darcy’s proposed objectives and outcome measures:

OBJECTIVES:

  1. Increase the grade point average for 60% of youth participating in the program.
  2. Decrease the number of school absences by at least two days per quarter for 90% of participating youth by program completion.
  3. All program youth will begin to participate in at least one school sponsored during the course of the program.

 

Notice that Darcy has specified these objectives with the percent of youth she expects to show change.  She can adjust these percents later to further improve the program’s success.  For example, Darcy feels it is reasonable to expect 60% of youth to increase GPA by the end of the program, but the following year, she may increase that number to 75% if the program is a success.

The initial and intermediate outcome measures listed below are directly linked to program objectives, and can be measured during or at the end of the program.

OUTCOME MEASURES:

  1. Number and percent of youth with increased grade point average by completion of program.
  2. Number and percent of youth with at least two fewer absences in the final quarter of the program compared to the first quarter of the program.
  3. Number and percent of youth participating in at least one school-sponsored activity during the course of the program.

 

Previous  Next

Step Four: Identify Measures

OUTCOMES AND PROGRAM LOGIC

Each outcome that we expect the program to affect is directly related to the program’s theory or logic.  The program goal is the long-term outcome anticipated by the program.  Let’s look at Darcy’s theory about her mentoring program to see how initial, intermediate and long-term outcomes are related to program logic.

Let’s take a look at Darcy’s first IF-THEN connection:

  • IF youth develop positive relationships with mentors who help with homework and studying, monitor attendance at school, and encourage and facilitate participation in extracurricular activities, THEN youths’ grades will improve, attendance will improve, and participation in extracurricular activities will increase.

 

All of the elements in the THEN statement (improvements in grades, attendance, and increased participation in extracurricular activities) are initial outcomes. These are the direct results of the mentors’ activities that we expect to see first.

 

Previous  Next

Step Four: Identify Measures

OUTCOMES AND PROGRAM LOGIC

Here are two more IF-THEN statements that follow Darcy’s program logic:

  • IF youths’ grades improve, attendance improves, and participation in extracurricular activities increases, THEN youths will feel stronger bonds or ties to their schools.

In Darcy’s program theory, improvements in school performance result in increased bonding to school. Youth bonding to school is therefore an intermediate outcome. It is a result of the initial outcomes, and is itself necessary for the program to meet its goal.

  • IF youths feel stronger bonds or ties to their schools, THEN they are more likely to adopt other prosocial values, such as a belief in the importance of lawful behavior.

The adoption of prosocial values other than the importance of academic achievement is an intermediate outcome that is a necessary step in achieving Darcy’s program goal.

Previous  Next

Step Four: Identify Measures

OUTCOMES AND PROGRAM LOGIC

Finally, here is Darcy’s last IF-THEN connection:

  • IF youths adopt prosocial values and believe in the importance of academic achievement, THEN they are less likely to engage in delinquent behavior. 

The decrease in the likelihood of delinquent behavior is a long-term outcome. This outcome is the ultimate result the program hopes to achieve; that is, the program’s GOAL.

Previous  Next

Step Four: Identify Measures

Darcy’s program operates within a larger system.  The last step in developing a logic model is identifying those factors external to the program that may affect whether the program will be able to achieve its goal and objectives.

 It is important to remember that a true evaluation, not just performance measurement, is needed to demonstrate that results have been caused by your program and not external factors.

Check out this logic model diagram with definitions for each of these components. 

 

 

This is the end of Module Three!  Click “Continue” to move on to Module Four.

Continue to the Next Module →

Previous  Next