Kennedy School Saguaro Seminar

PROGRAM EVALUATION GUIDE
Phase One: "PLANNING"

OVERVIEW

STEP 1: Mobilizing Resources

STEP 2: Understanding Social Capital as it Relates to Organizational Mission

STEP 3: Identifying Program Links to Social Capital

Once the organization, in its social capital retreat has restated its mission in social capital terms, it can move on to an examination of the specific program or programs to be evaluated. Staff and volunteers who work directly with participants and the public must be very involved in this step. We recommend that you bring a diverse group of stakeholders together for your social capital retreat. While this diversity of thought may make it more difficult to conduct such retreats and manage the conversation, it increases the chance that you will not miss out on an important observation. The ultimate decision as to what you want to evaluate is yours, and this may be important to identify up-front, but you will benefit from diverse perspectives.

We expect that program specifics will be discussed in a second meeting that may be held a week or more after the first meeting or meetings that dealt with the issue from an organization-wide perspective. If the members of the organization cannot answer some of the questions that follow, it may be necessary to undertake a small internal "study" of program operations.

Note: If your organization operates only one program, the thinking just done in Step 2 may significantly overlap with this step.

PROGRAM'S RELATION TO COMMUNITIES SERVED (expand)

SOCIAL CAPITAL FORMATION STRATEGIES (collapse)

    Task 2: Next, identify the ways in which program1 activities build social capital.

    The first question is how the program's services, activities, and events are intended to meet the organization's overarching social capital creation goals. We identify a number of basic approaches, broken roughly into direct and indirect social capital building:

    1) DIRECT SOCIAL CAPITAL BUILDING

    A community-based organization can create social capital directly through its programs. While by no means an exhaustive list, here are some basic ways in which programs might do this:

    Fostering Social Ties. If a program creates new social ties or strengthens existing ones it may directly create social capital among those who come into contact with it. (As you'll remember, we are particularly interested in new or stronger ties that expand the scope of information the individual is exposed to, provide stronger emotional support and/or facilitate increased trust and reciprocity.)

    To consider whether a program does this, put yourself in successive turns in the shoes of a typical member of various groups (staff member, program participant, project partner, parent of participant, etc.) and ask yourself whether he/she is likely to come out of this experience with significantly stronger ties. Look for repeat encounters and opportunities to build trust. Before conducting this step, you may want to even conduct some informal focus groups of one or more of these audiences, learning more about their experiences with the program.

    The planning group needs to inventory all the social interactions embedded in the program and assess them in terms of their potential social capital effects. Because this may be a new way of seeing the program, we hope that the organization's team takes the time to really "walk through" the program, looking for every opportunity for meaningful interactions. Once you've identified the broader inventory of what links are created you can decide where to focus in the "Which Ties?" step. It is also our hope that a side benefit of organizations undertaking this review will be to identify new opportunities to shift program structures or resources in order to diversify and extend the interactions among participants, their families, staff, and the community.

    Community Coalition-Building. An important variation on the above theme involves efforts to act directly on the community as a whole. As with the analysis above, it may be helpful to inventory all the coalition-building initiatives and analyze their potential social capital effects. Examples of this type of coalition-building include programs that directly organize or mobilize residents, that organize community-wide events, or that work with voluntary community associations to increase their effectiveness.

    Individual Awareness of Social Capital. Some programs directly try to make participants aware of social capital and try to develop more social capital through broader networks and civic participation. Sometimes this is a sub-part of a larger program goal (a job search, for example), and sometimes this awareness is itself a program goal (a voter registration drive, for example). In these cases, program evaluation may include an assessment of how well the message was retained as well as how participants actual behavior and social networks changed as a result.

    2) INDIRECT SOCIAL CAPITAL BUILDING STRATEGIES

    Organizations may conversely try to increase community social capital indirectly by dismantling barriers to social capital formation or creating new community resources that will facilitate neighborhood organization and sociability. Here are two ways in which programs might indirectly create social capital:

    Removing Barriers. Because social capital is rooted in simple social interactions, ranging from ordinary socializing to more formal group activities, anything that keeps people apart, thus limiting their interactions, also limits new social capital formation. If an organization successfully identifies and mitigates one or more barriers to social capital formation, it may help build more social capital. The most common barriers to social capital are safety and discrimination.

      Physical safety is a basic human need, and the fear of violence can easily keep people off streets and out of parks and make them wary of intervening when children misbehave or minor neighborhoods disputes arise. If cooperative efforts, additional or changed police protection, new lighting, private security, or other changes make people feel safer, this may well lead to more time spent outdoors together and more social capital created. The process of enhancing safety (such as through a neighborhood crime watch) can also enhance community collective action.

      Exclusion of groups, discrimination, and segregation represent the real manifestations of community divisions that community-building efforts seek to overcome. While a direct approach to building bridging social capital might bridge people from different groups together for a common purpose (e.g., wealthy and poorer residents working together on a park, or black and white kids on the same basketball teams), an indirect approach might identify and publicize patterns of exclusion, or seek changes that will make public spaces more inviting for groups who now feel uncomfortable there.

    Enhancing Community Resources. Another strategy to create social capital involves enhancing community resources underpinning social interactions. These resources build opportunities for more social capital just as road improvements support new business expansion. Here are a few ways in which programs can build the foundation for activities that create social capital:

      Creating and Reviving Public Spaces. The creation or revival of public spaces where people can meet and visit regularly is a prime example of building social capital infrastructure2. A new playground or community center may become a gathering place for parents and children. Public spaces like parks, libraries, schools, museums, public meeting rooms, gyms, flea markets, farmer's markets, the post office, and even the town dump can become places for regular social interactions and places where people learn about their community and what's going on in it.

      Creating and Reviving Civic Events. Closely related to public spaces are public events, like parades and celebrations. Shared experiences, like shared spaces, help create a sense of community that may help foster deeper forms of civic engagement.

      Improving Neighborhood Information Sources. The lack of information about one's community may exclude civic participation, so entities like neighborhood newspapers and community cable television programs may be forms of social capital infrastructure.

      Flextime and Release-time for Volunteering. Freeing up time for workers to participate in work-related social activities or to volunteer in the community similarly represents a resource that can be tapped to create new social capital.

    This non-exhaustive list of examples shows how a new resource can improve the possibility of social capital creation. Since those creating these resources expect that increased social capital will be an outcome (whether they label it "social capital"), it is certainly reasonable for them to measure social capital. It is also possible that other basic changes in a community - a shift in the housing mix, a business boom, for example - may ultimately enhance social capital. However, unless community organizations have special reasons to look for these second order effects, we do not recommend including them in program evaluation design.

    [Note: Direct and indirect interventions could also be combined in the same program; for example, revitalizing a community center using considerable neighborhood participation might both build social capital in the process and create social capital infrastructure (the community center) which indirectly increased social capital.]

    Ongoing example: check in with Jumpahead

1. Throughout, we will use the term program in a generic way to refer to any integrated set of one or more services, activities, and events through which a community organization achieves its mission. Even given the dizzying array of such programs we will try to make some generalizations about social capital.
2. A new public safety initiative may reclaim these spaces, so our categories of "removing barriers" and "creating public space" are closely related.

WHICH TIES ARE ORGANIZATION TRYING TO BUILD? (expand)

IS SOCIAL CAPITAL AN ESSENTIAL PART OF PROGRAM OPERATIONS? (expand)

Go on to Phase Two

 

 

TABLE OF CONTENTS

INTRODUCTION

PHASE ONE | Planning

  • Step 1: Mobilizing Resources
  • Step 2: Understanding Social Capital as it Relates to Organizational Mission
  • Step 3: Identifying Program Links to Social Capital

PHASE TWO | Evaluation

  • Step 4: Designing the Evaluation
  • Step 5: Conducting an Evaluation

PHASE THREE | Action

  • Step 6: Interpreting the Results
  • Step 7: Revising Programs

GLOSSARY


This guide was created by
Thomas Sander, Executive Director of the Saguaro Seminar, &
Stephen Minicucci, Ph.D.,
Principal Investigator

Edited and adapted for the web by Benjamin Toff

E-mail us your ideas for improving this Guide.


SEARCH THE SAGUARO WEBSITE
home | about Social Capital| contact us | email us | KSG | Harvard | site map

Copyright, 2007, President and Fellows of Harvard College

website design by WhiteDogDesign.com

go to main harvard site