Once an organization has identified the social capital elements in its programs and specified pathways through which its activities contribute to social capital formation, it can formulate plans to evaluate its social capital impact. Somewhat artificially, we break down this process into steps 4 and 5: Step 4 involves the basic design questions, such as the study's timeframe and the choice of study populations; Step 5 concerns the nuts and bolts of fielding a survey-based evaluation. You can also access a list of social capital survey questions intended to help organizations develop their own survey instruments.

STEP 4: Designing the Evaluation

STEP 5: Conducting an Evaluation

There are a number of concrete issues that an organization fielding an evaluation still needs to resolve. This section considers five key practical questions listed below.

Together with the earlier "Whom to Study?" discussion, the answers to these questions largely determine the overall evaluation cost. Of course, if the resulting cost is too high, an organization may have to scale back, compromising on these four issues until the project is affordable. At that point, the organization will have to ensure that the scaled-down evaluation will still be worthwhile.

INTERVIEW FORMATS (expand)

HOW MANY CASES? (expand)

MAKING SURE SAMPLES ARE RANDOM (AND NON-RESPONSE ERROR) (expand)

WHO SHOULD CONDUCT THE EVALUATION? (collapse)

Larger organizations with an internal evaluation or research department should probably use such a department for the study. Smaller organizations typically need external help for Steps 4-6 (the "Evaluation" phase and "Interpreting the Results"), but all organizations should be actively involved in Steps 1-3 (the "Planning" phase), even if using an outside evaluator. If you want guidance on how best to select an external evaluator, see the W.K. Kellogg Evaluation Handbook chapter 5, step 4 (pp. 57-68).

Interviewing represents the most time-intensive element of the evaluation process. Most interviews will take 15-30 minutes, permitting only 2 surveys to be conducted per hour on-site or by phone, and fewer off-site. The time necessary to field 100 or 200 interviews quickly mounts. (Also bear in mind that you often need to call, for example, 300-600 numbers to get 100-200 willing respondents.) We outline below a number of options for staffing the interviews. There are only two restrictions: (1) interviewers should not have had significant prior contact with the respondents (or it could upset the objectivity of responses); and (2) they should be trained in the survey instrument (i.e., asking the survey questions).

Training
All interviewers should go through a short training in which they are introduced to the basic goals of the survey (without biasing interviewers by telling them your preconceived expectations), go over common points of difficulty with the instrument, and interview one another for practice. Note: this is the first occasion that the evaluation team will have to pre-test the instrument: i.e., see if respondents find question wording vague or unclear, see if many respondents want to answer with an option that is not listed, etc. If you have to replace interviewers or substitute in new ones in the midst of your survey, make sure all new interviewers are trained in this same way.

The interviewers should also each conduct one or two pre-test interviews on qualified respondents before interviewing the actual population being surveyed. The survey is conducted as if it was real, but the results are discarded and not entered.

Interviewers also need to understand what confidentiality procedures exist for respondents.

Ongoing example: hear about Jumpahead's sample size, sampling approach, and who conducted the evaluation

HOW TO CONSTRUCT THE QUESTIONNAIRE (expand)