There’s a binder on the top shelf of the bookcase in Dan Levy’s Kennedy School office. It’s a reminder, says Levy, a lecturer in public policy, of when he arrived at the Kennedy School as a visiting faculty member eight years ago, after years working as a policy researcher. He was attracted by the school’s energy and moved by a passion to teach.
But his arrival was not what he might have expected. “When I first came, I was told, “Here are the three courses you’re going to teach; here is the syllabus from last year; good luck,” Levy remembers with a laugh. “The first course I taught went terribly.”
He got help from other teachers but there was not much else. Since then, Levy has dedicated himself as much to the art and science of teaching as to his public policy work, becoming what many on campus describe as a “master teacher,” one of the school’s best classroom teachers.
But he doesn’t let that go to his head. In that binder on the top shelf are student evaluations of Levy from his first class. They were not kind. “Whenever I’m feeling good about my teaching, I take them out and humble myself by reading what students said about me,” Levy says.
Levy is one of three faculty cochairs of Strengthening Learning and Teaching Excellence (SLATE), an initiative launched by the school to improve education at HKS. Since 2007, SLATE has worked on helping teachers teach and finding out what and how students are learning.
It started with a survey.
The results from a huge 2005 survey of Kennedy School alumni surprised school administrators. When the former students were asked what type of teaching they had found most useful, a large majority cited case studies. But the school’s Case Program had fallen on hard times. Because of a lack of funds, case studies were sometimes produced in cooperation with outside backers, so not only were fewer being created, but their relevance to what was being taught at the school was decreasing.
Dean David Ellwood and Mary Jo Bane, then the academic dean, asked Jack Donahue MPP 1982, Vernon Senior Lecturer in Public Policy and faculty chair of the MPP program, to serve as faculty chair of the Case Program — the first since Richard Neustadt — to help chart the program’s future. Soon he was asked to work on teaching, too.
Donahue agreed, but on condition that he could find cochairs: Levy and Richard Light, Gale Professor of Education at the Graduate School of Education, an adjunct faculty member at HKS, and one of the leading statisticians working on the problems of American higher education. An advisory committee made up of campus leaders and senior faculty is chaired by Derek Bok, President Emeritus of the university, and a long-time leader in pedagogical efforts at Harvard.
“The Kennedy School was born innovative,” says Donahue about the school’s “rebirth” in the late 1960s, under the aegis of giants such as Tom Schelling, Richard Neustadt, Frederick Mosteller, and Howard Raiffa. “It was a bunch of brilliant misfits who didn’t like either the content or the method of teaching in arts and sciences and set up something different. They in turn would train each incoming generation in that rule-breaking style.
“Well, the school got big, the founding fathers died or retired, and for a while we didn’t notice that we didn’t have the organic capacity to transmit that culture. Then, at the start of Dean David Ellwood’s tenure, we realized we’d lost that and needed an institutional capacity to do what Neustadt and Raiffa used to do.”
What Neustadt and Raiffa did — how their students learned and how that knowledge was imparted — was not easy to determine. But recent advances in cognitive science and more focus on the art of teaching itself have helped.
Peer instruction is part of a broader category often referred to as active learning, an approach built on understanding how knowledge is absorbed and then used effectively. Peer instruction aims to tackle the misconceptions that students bring to the classroom, so as to develop a proper foundation of factual knowledge, that they can then develop into contextual frameworks and organize in a way they can remember and apply.
Once these building blocks are in place, the student can begin to take control of his own learning, knowing where he stands and what needs improving — a process known as metacognition.
Applying all that to the classroom is challenging; especially at the Kennedy School, where students are at the intersection of theory and practice.
“Our students are going to apply what they learn to solve problems in the messiness of the real world,” says Levy. “Statistics may be an important input into a problem, but then they’re going to have to form a coalition, see what stakeholders they’re going to bring in, manage a process, perhaps think about the economics behind it.”
Technology provides some help. Kennedy School teachers are increasingly turning to “clickers,” remote control–like devices that students use in the classroom to respond to questions in real time. Levy remembers a moment when he realized how important these could be. He was giving students in his statistics class a “warm-up” question (What is the probability that if you roll a pair of dice twice, you get at least one six?), which he expected about 80 percent of his students to get right. Only 17 percent did.
He remembers freezing for a minute in class, unsure of how to proceed. Then, seeing that almost half the class had made the same mistake, he was able to correct their misconceptions (known in statistics as double-counting) and move on.
“This was a moment I realized the importance of constantly gauging where my students are,” Levy says.
The Kennedy School has also received a grant from the Harvard Initiative on Learning and Teaching (a university-wide effort to catalyze innovation in teaching) to leverage technology to make the most effective use of classroom time. Moving standard content out of class time and into multimedia, explains Suzanne Cooper, associate academic dean and Stokey Senior Lecturer in Public Policy, can free up scarce classroom time to make room for activities that promote more active learning.
The “flipped classroom,” as it is known, will use short video lectures, online tutorials, and practice tests to leave class time for peer-to-peer interactions, experiential learning, and exercises that integrate across courses to teach more realistic problem-solving, Cooper says. Client-based projects (see Deep Learning, below) are among these active learning approaches.
But more important than any single technique or technology is a culture of teaching at the school. SLATE has tried to provide tools to support faculty members in their teaching — in an effort, as Donahue describes it, “to have an institutional capacity to encourage and promote and promulgate innovations in pedagogy.”
SLATE has created a New Faculty Institute, which each summer gives new faculty members a three-day orientation that includes practice teaching sessions, observations of “master teachers” in the classroom, and an overview of strategies, resources, and policies. Staff members Allison Pingree, director of professional pedagogy, and Josh Bookin, curriculum solutions specialist, can help teachers design a curriculum to advance carefully determined learning objectives.
“It’s ideal to focus on a course in the early stage — like preventive medicine instead of triage,” says Pingree, who worked at Harvard’s Bok Center for Teaching and Learning and Vanderbilt’s Center for Teaching before coming to the Kennedy School in 2011. “A lot of the problems faculty experience in their teaching come from a lack of foundational design work early on, to clarify and articulate what they want their students to learn.”
There’s a natural tendency to focus more on the content to be covered than on the teaching of it. But frustrations that students may experience — such as not understanding what’s expected of them, how they will be graded, or the inter-relationship of class sessions is — can often be eliminated by building the curriculum on what students should learn and how that learning will be assessed, and then fitting each session into that larger picture, Pingree explains.
SLATE has also instituted HKS Teaching Week. Each term for the past two years, faculty members have opened their classrooms to observation by their colleagues (this past spring, 22 classes participated — nearly 20 percent of all classes taught at the school) in an attempt to demonstrate what they are doing and share reflections and suggestions among peers. And faculty lunches offer structured yet casual opportunities to discuss teaching (see “Faculty Mentoring,” below).
“Teaching — in particular, the same topic — can easily become routine,” says Pinar Dogan, a lecturer in public policy, who has worked with SLATE on a number of initiatives, including a focus group with former students. “We tend to seek improvement only if things go somewhat wrong, which limits our ability to produce effective preemptive improvements in our teaching and courses. SLATE has created a platform where faculty can get together and brainstorm about effective teaching.”
Part of SLATE’s mission has also been to revamp the Case Program.
“The Case Program leads with the learning objective rather than leading with the story,” says Carolyn Wood, assistant academic dean and director of SLATE. “We ask, ‘How can this case set up a rich discussion in the classroom?’ rather than just thinking, ‘This is a really engaging story — let’s write it up.’”
Case studies are produced only when faculty members express a desire to use them in a class. Three case writers and a small video and multimedia unit are now on the team, thanks in part to a gift from Joseph Tompkins MPP 1975 (see “Medium and the Message,” below).
How effective has all this new activity been? Answering that question is one of the most compelling parts of SLATE’s mission.
Richard Light, SLATE’s third faculty cochair is responsible for assessing learning at the school. “How do we as a faculty know that our students are learning how to think, how to approach new problems, how to be leaders in the public sector, how to solve problems in a constructive, rigorous, analytic way?” he asks.
To answer, SLATE has been conducting a number of assessments to understand what Kennedy School students are learning. Testing begins with individual courses. Students are given the same questions at the beginning and then at the end of the course. They take the test anonymously, identified only by a code number that will allow the pre- and post-course tests to be matched. The answers are graded blind by faculty members or teaching assistants who are not aware of the students’ identities or whether the tests were taken at the beginning or the end of the year.
The assessment has expanded to 10 individual degree program courses and two executive education courses. In the 2010–2011 academic year, the entire first-year MPP class was assessed, and this past year the Social and Urban Policy concentration cohort of the MPP class was tested.
Repeating the testing allows Light and the SLATE team to refine their approach. It’s relatively easy to gauge whether students are learning basic factual knowledge, but it becomes increasingly complicated to measure whether they can transfer that knowledge to other disciplines or combine all their knowledge on a broad, open-ended question of public policy.
The results, which have been shared with faculty members and the Kennedy School leadership, have been encouraging by most measures but disappointing by others.
More important, they will serve as a foundation on which future learning can be gauged and, possibly, linked to innovations in teaching at the school.
“It’s not like we’re starting something that everybody knows how to do,” Light says of SLATE’s work. “If you went to other great universities now and asked, ‘What are you folks doing to gauge learning?’ the answer would be: ‘Not much, because it’s more difficult than it looks.’ The goal is not to say that we’re better than them, but rather to say, ‘We’re charting new territory; we hope it works.’”
Medium and the message
When it comes to case studies, print is king. A written presentation is ideal when transmitting lots of detailed information to students. But other media are increasingly being used. In spring 2012, the Kennedy School’s first Web-based multimedia case was used in Archon Fung’s Sparking Social Change class (cotaught with Mark Moore). Fung approached Jack Donahue and Carolyn Wood of SLATE with the idea of doing a case on coordinated abandonment (a theory developed by Gerry Mackie of the University of California at San Diego to explain very rapid social change). The case would be about a campaign, led by an NGO, to end female genital cutting in the West African nation of Senegal. The issue is one that elicits strong emotional reactions, and is often met with paternalistic or simplistic approaches. So the decision was made to create a Web-based case study, that would include numerous video interviews with people involved with it. “We felt it was very important to have the people in the community tell the story,” says Patricia Garcia-Rios, SLATE’s multimedia case producer. “We needed to give students something that had texture, that they could relate to, that allowed them to immerse themselves in those communities.” Students concurred. Layusa Isa-Odidi MPA/ID 2013 responded to the filmed interview of a community’s imam describing how women who had not been cut would be ostracized. “Certain points are difficult to get across in writing,” she said. “You can read ‘This is a very stigmatized practice,’ but to have the imam sort of lay out ‘If this woman wasn’t cut . . . if she would cook food, we wouldn’t eat it,’ that was very powerful.”Quality reasoning
Assessing learning is never easy — especially when, as at the Kennedy School, education is geared toward real-world practical application. Students are expected not only to master specific subjects and then apply that learning in other areas, but also to know when to apply what they’ve learned to open-ended questions that may have no single right answer. In the 2010–2011 academic year, the SLATE team administered the same test twice to first-year MPP students: at the beginning of the year and at the end. The question at right is one to which students could give “dozens of good and thoughtful answers,” says Richard Light, SLATE faculty cochair. “We looked when grading for rigorous thinking, and faculty members made a list of points that ideally students might mention. So students could write an answer that recommends option ‘A’ or they could recommend option ‘B’ — and for our grading we focus entirely on the quality of their reasoning and whether they hit several of the points faculty listed from their classes, which could buttress a compelling argument in favor of an alternative. That is precisely what makes this a good question.”
The Scenario You are a special assistant to Josette Sheeran, the executive director of the United Nations World Food Programme (WPF). The WPF uses two different approaches to reduce hunger and malnutrition worldwide. One approach is to acquire food — by either purchase or donation — from countries with food surpluses. The WPF then transports this food to countries with immediate need and where there are malnourished people. The second approach involves efforts at development, to help countries attain self-sufficiency through improvements in local capacity to grow, to process, and to distribute food.
The director wants your advice on a systematic framework for how the WPF should determine the balance between these two approaches within each country where it operates.
Question What are the five most important pieces of evidence that a World Food Programme mission in a country should track to determine the best balance between these two approaches — food donations versus assistance toward food self-sufficiency — within a particular country? In other words, what factual matters are most relevant to this strategic choice — and, in a few words, why is each one relevant?
At a Kennedy School faculty meeting last year, Eric Mazur, Harvard physics professor and pedagogy guru, shared his insights on teaching. Appropriately, he involved his audience in active learning. He asked them a question (If a metal plate with a hole in the middle is heated, will the hole expand, shrink, or stay the same size?) and then invited the faculty members to break out into small groups to discuss the answer. It was an example of peer instruction, and one that made an impression. Months later, faculty members gathered again, this time at an informal lunch organized by SLATE, to discuss the experience of those teachers who had incorporated peer instruction and clicker use in their classrooms. Richard Zeckhauser, Ramsey Professor of Political Economy, and Lant Pritchett, professor of practice and an award-winning teacher at the Kennedy School, led off the discussion.
The event was a perfect illustration of the culture SLATE is helping to create at the school: Zeckhauser, who learned his craft as an educator from some of the school’s “founding fathers” four decades ago and continues to innovate his teaching style, and junior faculty members such as Pinar Dogan, a lecturer in public policy who has used clicker technology extensively in her game theory class, shared insights about the best uses of technology.
Whatever new innovations and technologies are used, some time-tested truths always remain. “Make sure your teaching is a creative process,” Zeckhauser advised his colleagues.
Linda Bilmes’s Advanced Applied Budgeting class, with its emphasis on fieldwork and client-based learning (see Q+A) is a prime example of the sort of active engagement that allows students to maximize learning. This past spring, for example, some of Bilmes’s students conducted a financial assessment of Hubway, Boston’s bike-sharing program, and presented their findings to Boston Mayor Thomas Menino. According to Carolyn Wood, who worked alongside Professor Bilmes to manage the Advanced Applied Budgeting course for several years, providing students with an opportunity to apply the skills they’re learning in the classroom to real public problems — where there’s a chance their ideas will be implemented — is highly motivating for them. “They work incredibly hard and they learn on many different levels at once,” she says.
But this experience is not easily replicated. Experiential learning courses require faculty members to shift out of content-delivery mode and into coaching mode, which takes some getting used to. Recruiting strong clients and stewarding those relationships takes time and effort, and shaping projects that are realistic, fit neatly into a semester schedule, and enable students to integrate and apply the skills learned in their foundation courses can be challenging. SLATE can help teachers tackle those challenges by capturing and disseminating lessons from effective experiential courses and assisting faculty members to support team-based and project-based learning.