Kennedy School Saguaro Seminar

PROGRAM EVALUATION GUIDE
Phase Two: "EVALUATION"

Once an organization has identified the social capital elements in its programs and specified pathways through which its activities contribute to social capital formation, it can formulate plans to evaluate its social capital impact. Somewhat artificially, we break down this process into steps 4 and 5: Step 4 involves the basic design questions, such as the study's timeframe and the choice of study populations; Step 5 concerns the nuts and bolts of fielding a survey-based evaluation. You can also access a list of social capital survey questions intended to help organizations develop their own survey instruments.

STEP 4: Designing the Evaluation

STEP 5: Conducting an Evaluation

There are a number of concrete issues that an organization fielding an evaluation still needs to resolve. This section considers five key practical questions listed below.

Together with the earlier "Whom to Study?" discussion, the answers to these questions largely determine the overall evaluation cost. Of course, if the resulting cost is too high, an organization may have to scale back, compromising on these four issues until the project is affordable. At that point, the organization will have to ensure that the scaled-down evaluation will still be worthwhile.

INTERVIEW FORMATS (collapse)

    There are at least six basic formats, although combinations are also possible:

    • In-person interviews conducted on-site
    • In-person interviews conducted off-site (in the person's home, at school, etc.)
    • Self-administered questionnaires answered on-site
    • Telephone interviews
    • Mail or e-mail surveys
    • Interactive or Internet evaluations (using a computer program).

    Each of these models has advantages and disadvantages. [If you are undecided about what approach you will take and want to read about the relative strengths and weaknesses of these approaches, click here.]

    Ongoing example: hear about Jumpahead's interview format

HOW MANY CASES? (expand)

MAKING SURE SAMPLES ARE RANDOM (AND NON-RESPONSE ERROR) (expand)

WHO SHOULD CONDUCT THE EVALUATION? (expand)

HOW TO CONSTRUCT THE QUESTIONNAIRE (expand)

 

 

TABLE OF CONTENTS

INTRODUCTION

PHASE ONE | Planning

  • Step 1: Mobilizing Resources
  • Step 2: Understanding Social Capital as it Relates to Organizational Mission
  • Step 3: Identifying Program Links to Social Capital

PHASE TWO | Evaluation

  • Step 4: Designing the Evaluation
  • Step 5: Conducting an Evaluation

PHASE THREE | Action

  • Step 6: Interpreting the Results
  • Step 7: Revising Programs

GLOSSARY


This guide was created by
Thomas Sander, Executive Director of the Saguaro Seminar, &
Stephen Minicucci, Ph.D.,
Principal Investigator

Edited and adapted for the web by Benjamin Toff

E-mail us your ideas for improving this Guide.


SEARCH THE SAGUARO WEBSITE
home | about Social Capital| contact us | email us | KSG | Harvard | site map

Copyright, 2007, President and Fellows of Harvard College

website design by WhiteDogDesign.com

go to main harvard site