Kennedy School Saguaro Seminar

PROGRAM EVALUATION GUIDE
Phase One: "PLANNING"

OVERVIEW

STEP 1: Mobilizing Resources

STEP 2: Understanding Social Capital as it Relates to Organizational Mission

STEP 3: Identifying Program Links to Social Capital

Once the organization, in its social capital retreat has restated its mission in social capital terms, it can move on to an examination of the specific program or programs to be evaluated. Staff and volunteers who work directly with participants and the public must be very involved in this step. We recommend that you bring a diverse group of stakeholders together for your social capital retreat. While this diversity of thought may make it more difficult to conduct such retreats and manage the conversation, it increases the chance that you will not miss out on an important observation. The ultimate decision as to what you want to evaluate is yours, and this may be important to identify up-front, but you will benefit from diverse perspectives.

We expect that program specifics will be discussed in a second meeting that may be held a week or more after the first meeting or meetings that dealt with the issue from an organization-wide perspective. If the members of the organization cannot answer some of the questions that follow, it may be necessary to undertake a small internal "study" of program operations.

Note: If your organization operates only one program, the thinking just done in Step 2 may significantly overlap with this step.

PROGRAM'S RELATION TO COMMUNITIES SERVED (collapse)

Task 1: Define How the Program Expects to Affect The Relevant Communities

Whether the program, for example, expects to impact any random citizen in a certain geographic area, or only a program participant matters a great deal in considering how to evaluate the social capital impact of a program. Measurement and evaluation choices hinge on the visibility of a program and the scale of its impact. Community health organizations, general education and youth services programs, community advocates and organizers, and anti-poverty agencies, among others, typically operate highly visible local efforts serving relatively small geographic communities. Community health centers and youth development agencies (like the YMCA or Boys and Girls Clubs) aim to be a part of the everyday life of their communities. Other programs, like community development corporations, value visibility but don't directly touch many residents' lives as often. Because of their institutional position in community life, it is reasonable for these organizations (like the YMCA or the Boys and Girls Clubs) to be concerned with creating social capital in the geographical community served, and not simply among program participants. Other programs operate on a broad geographic scale but involve a small fraction of residents. There are at least two such types of programs:

i) Programs Serving a Disparate Population: Many arts and cultural institutions fit this model, as do programs that organize, coordinate, or support a number of different local organizations. These programs have a much more diffuse impact on community social capital (defined geographically) and therefore the social capital impact is harder to measure. This does not mean that they cannot contribute to the creation of social capital. The events and activities they sponsor may have a memorable impact on the participants' lives even if they don't affect a random community member in that town.

City- and region-wide programs can create bridging social capital in ways that neighborhood and town programs cannot. Obviously, it would be asking a great deal to expect these activities to measurably increase social capital in the whole community they serve (of perhaps one million or more!). Instead, the evaluation should likely focus on how these programs change how individual participants relate to the particular communities of which they are part.

ii) Programs providing special services: Second, programs can also provide special services tailored to relatively narrow population groups. They might provide this service in a centralized location (a large residential facility or a special school, for example) or operate small-scale programs in varied communities.

While programs may have a number of social capital goals, their most likely aim is to serve their program participants heavily -- whether that service is participation in an extra-curricular arts program, a homeless shelter, or a choral group. Because the population served may comprise only a small part of a whole neighborhood or town, the effect that these programs is likely to be seen on the program participants, not on the social capital of the community as a whole.

Ongoing example: check in with Jumpahead.

SOCIAL CAPITAL FORMATION STRATEGIES (expand)

WHICH TIES ARE ORGANIZATION TRYING TO BUILD? (expand)

IS SOCIAL CAPITAL AN ESSENTIAL PART OF PROGRAM OPERATIONS? (expand)

Go on to Phase Two

 

 

TABLE OF CONTENTS

INTRODUCTION

PHASE ONE | Planning

  • Step 1: Mobilizing Resources
  • Step 2: Understanding Social Capital as it Relates to Organizational Mission
  • Step 3: Identifying Program Links to Social Capital

PHASE TWO | Evaluation

  • Step 4: Designing the Evaluation
  • Step 5: Conducting an Evaluation

PHASE THREE | Action

  • Step 6: Interpreting the Results
  • Step 7: Revising Programs

GLOSSARY


This guide was created by
Thomas Sander, Executive Director of the Saguaro Seminar, &
Stephen Minicucci, Ph.D.,
Principal Investigator

Edited and adapted for the web by Benjamin Toff

E-mail us your ideas for improving this Guide.


SEARCH THE SAGUARO WEBSITE
home | about Social Capital| contact us | email us | KSG | Harvard | site map

Copyright, 2007, President and Fellows of Harvard College

website design by WhiteDogDesign.com

go to main harvard site