Once an organization has identified the social capital elements in its programs and specified pathways through which its activities contribute to social capital formation, it can formulate plans to evaluate its social capital impact. Somewhat artificially, we break down this process into steps 4 and 5: Step 4 involves the basic design questions, such as the study's timeframe and the choice of study populations; Step 5 concerns the nuts and bolts of fielding a survey-based evaluation. You can also access a list of social capital survey questions intended to help organizations develop their own survey instruments.

STEP 4: Designing the Evaluation

The design of the evaluation hinges on answering a sequence of questions discussed below. The answers to these questions flow directly from the earlier discussion of what the program is trying to do.

WHO IS THE STUDY GROUP? (collapse)

Which population will be the focus of the evaluation effort? Since the members of the organization may feel that they have a positive impact on the social capital of a wide variety of different groups, choosing one or more group for special examination might be difficult. [You can choose multiple "populations", such as parents of program participants and program participants themselves, but each additional population chosen will increase the cost and complexity of the evaluation so you may want to stick with one group and you probably don't want to choose more than two or three maximum.]

In the evaluation-planning phase, we strove for completeness and worked mainly in the realm of ideas and words. Now, as we shift to the practical problem of fielding an actual evaluation, concerns about cost and practicality emerge. An organization may find that it cannot afford or manage the ideal evaluation of its program. This does not mean the whole effort is wasted; we can gain tremendous insight from a "second-best" evaluation implemented with full knowledge of its limitations. A less intensive and less conclusive evaluation may also form the basis of a proposal to fund a more sophisticated evaluation.

Task 1: Specify a study group.

From your Step 3 inventory, look at groups of individuals whose direct and indirect ties you thought were critical to impact. Depending on the program, these might be, for example: program participants and/or their families, staff, volunteers, residents of a certain neighborhood, etc. If only one group appears on this list, this decision is straightforward. If two or more critical groups appear, the situation becomes more complicated and you may well have to make choices. Because of cost or other considerations, an organization may decide not to ask questions of both volunteers and program participants, for example. In this case, the evaluators may choose to focus on one group at a time. Later, we consider how the same evaluation can include different sets of respondents. The group whose social capital we decide to measure is called the study group.

For most purposes, program participants and/or their immediate families are the appropriate group to study, but there are certainly cases in which the focus should be staff or volunteers, or specific outside groups with which the organization interacts, such as members of a local crime watch group. Sometimes organizations may also aim to increase social capital among a much wider audience. For example, groups that organize large community events or serve a large percentage of the neighborhood (the community-health center, or a major youth center such as the YMCA or Boys Club). In these cases, a general sample of the neighborhood or community (however defined) may make more sense to study. We'll make a general distinction here between the latter, a community sample, and all of the former cases, which we'll simply call program samples.

We'll worry later about the right size of this sample.

HOW CAN WE BEST IMPLEMENT A BASIC "BEFORE AND AFTER" STUDY FRAMEWORK? (expand)

SHOULD WE USE A COMPARISON GROUP? AND, IF SO, WHO? (expand)

WHAT SHOULD WE MEASURE? (expand)

STEP 5: Conducting an Evaluation