• Evaluation is about asking good, critical questions to help us learn and be accountable. Identifying “good” questions is an important aspect of creating useful evaluations.

      
  • What is important to measure?
  • What will you spend time and resources on?

• One of the greatest benefits of the logic model is that it clarifies what the program is. Understanding what the program is, is the first step in any evaluation.
• What, in particular, do you want to evaluate? Is the focus of the evaluation the whole program or a component of the program?

• Section 7: Using Logic Models in Evaluation

   

  <!-- .entry-header -->     

    

     
• In addition to helping you plan a program, the logic model can help you measure your outcomes and improve your evaluation practice. In this section you will learn how the logic model can help you determine what to evaluate, identify appropriate questions for evaluation, select indicators, know when to collect data and determine what data collection methods are most appropriate.
• • Recognize how the logic model helps determine what you will evaluate – the focus of your evaluation.
  • Recognize how the logic model helps you determine meaningful and useful evaluation questions – know what to measure.
  • Understand indicators and know what information best answers your evaluation questions.
  • Be able to identify appropriate timing for data collection.

1 more annotation...

• As you finalize your outcome chain and focus on the outcome(s) of interest, streamline your outcome chain by considering whether the outcomes are:

      
  1. Important: Are the end outcomes important? Do they represent significant change or improvements that are valued by participants and key stakeholders? Outcomes may be achievable but not really worth the effort. Apply the “Who cares?” test.
  2. Reasonable: Are the outcomes linked in reasonable order? Is it likely that one will lead to the next, which will then will lead to the next?
  3. Realistic: Are the outcomes realistic given the nature of the problem, your resources, and your abilities? Will the program lead to or help contribute to these outcomes? (Be careful to ensure that the outcomes are realistic given the level of effort.)
  4. Potentially negative/accompanied by negative consequences: What are potential negative effects that we need to anticipate? What else might happen? Or, how else might the sequence of events unfold?

• The environment in which the program exists includes a variety of external factors that can influence the program’s success. External factors include the cultural milieu, the climate, economic structure, housing patterns, demographic patterns, political environment, background and experiences of program participants, media influence, and changing policies and priorities. These external factors may have a major influence on the achievement of outcomes. We can’t ignore them! They may affect a variety of things including the following:

      
  • Program implementation
  • Participants and recipients
  • The speed and degree to which change occurs
  • Staffing patterns and resources available

• Assumptions are beliefs we have about the program and the people involved and the way we think the program will work. This is the “theory” we are talking about: the underlying beliefs in how it will work. These are validated with research and experience. Assumptions underlie and influence the program decisions we make. Assumptions are principles, beliefs, or ideas about:

      
  • The problem or situation.
  • The resources and staff.
  • The way the program will operate.
  • What the program expects to achieve.
  • The knowledge base.
  • The external environment.
  • The internal environment.
  • The participants: how they learn, their behavior, motivations, etc.
• • the resources are adequate and available,
  • a culturally appropriate curriculum can be developed and delivered effectively,
  • targeted parents are willing and able to attend,
  • and that knowledge change leads to behavior change.
    
     

  In developing a logic model, we want to make explicit all the implicit assumptions we are making. They may not all be portrayed in the one-page graphic, but we do want to explore and discuss them. Often, inaccurate or overlooked assumptions are the basis for failure or less than expected results.

• Clarifying assumptions demands knowledge of the research or “best practice” in the substantive area, as well as “common sense.”

      
   
  Consider the following:
      
  • Why do you believe that the program will work this way? Are your ideas and beliefs based on research, best practice, experience, local wisdom, intuition?
  • Is there evidence that supports the theory of action you’ve laid out?
  • Review the following:
    • Programming and change strategies that have proved effective in similar communities or situations
    • Research literature
    • Evaluation reports

1 more annotation...

• Outcomes are the direct result or benefits for individuals, families, groups, communities, organizations, or systems. Examples include changes in knowledge, skill development, changes in behavior, capacities or decision making, and policy development. Outcomes can be short-term, medium-term, or longer-term achievements. Outcomes may be positive, negative, neutral, intended, or unintended. Because outcomes are so central to logic models, they are discussed in more detail in Section 2.

• Outputs are the activities, services, events, and products that reach people (individuals, groups, agencies) who participate or who are targeted.

      

  Outputs are “what we do” or “what we offer.” They include workshops, services, conferences, community surveys, facilitation, in-home counseling, etc.

      

  These outputs are intended to lead to specific outcomes.

• Inputs are the resources and contributions that you and others make to the effort. These include time, people (staff, volunteers), money, materials, equipment, partnerships, research base, and technology among other things.

      

  These inputs allow us to create outputs.