What works in workforce development?

For Americans without a college degree, finding a job has become harder and harder over the last few decades. In some parts of the country, the joblessness rate for this group exceeds fifty percent. In those places, over half of working-age people without a college degree don’t have jobs.

Part of this change is due to job losses. Automation, globalization, and energy transitions have hollowed out industries and regional economies. However, this isn’t the whole story. New jobs are being created all the time, but employers aren’t filling these roles because they can’t find the right person for the job.

This is where workforce development comes in. Workforce development organizations train workers, provide wraparound services, and create connections between employers and prospective employees. Thousands of these public and private organizations operate throughout the country, and they’ve developed an impressive array of programs and networks for getting people into jobs. But, a huge open question remains: what works? How do we evaluate workforce development?

Last week, the Reimagining the Economy project, along with Marshall University and Shaping Our Appalachian Region (SOAR — a nonprofit focused on population growth in Eastern Kentucky), brought together workforce development practitioners from the tristate area of Ohio, Kentucky, and West Virginia to answer this question. This included representatives from community colleges, public universities, and non-profits with education and workforce development programs. They met on the Marshall University campus in Huntington, West Virginia to discuss topics from the finer details of data collection all the way up to big ideas of what matters and why. This event was the first of three planned workshops with stakeholders from West Virginia and Eastern Kentucky.

 

The Reimagining the Economy project aims to make an economy that works for everyone: repairing the economic systems that have left people and places behind and exploring new options to replace them. The project draws on academic research as well as insights directly from policymakers and practitioners.

“What we have discovered along the way,” says Gordon Hanson, the Peter Wertheim Professor in Urban Policy and co-founder of Reimagining the Economy, “is that the good designs are there. Practitioners have it figured out. The challenge is spreading these good ideas in places where they’re not being implemented and funding these ideas at a scale that can be transformational, not just in the lives of individuals, but in the lives of communities.”

“The challenge is spreading these good ideas in places where they’re not being implemented and funding these ideas at a scale that can be transformational, not just in the lives of individuals, but in the lives of communities.”

- Gordon Hanson

The evaluation question — the question of what works in workforce development — matters for workforce developers themselves in coming up with best practices, but it’s also a question for society at large. What types of programs should we have and how much public resources should be dedicated to them?

There are no easy answers. The complications of evaluation came up again and again in the discussions with the practitioners.

“When it comes to workforce development, there’s some very surface-level metrics that we all know,” said Brandon Dennison, Vice President for Economic and Workforce Development at Marshall University. “Like, did you create a job? Did you train a person? Maybe you can go a little bit deeper. Does that job pay well? Does that job have benefits? But really what I’m seeing is that ‘did that program work?’ is a very loaded question. Every single individual who goes through a training program or comes onto a campus is a unique individual who’s overcome a unique set of circumstances.”

“What I’m seeing is that ‘did that program work?’ is a very loaded question.”

- Brandon Dennison

In the areas of Appalachia the practitioners serve, it’s not uncommon for people to arrive at a workforce development organization having never held a job, struggling with addiction, or without electricity in their home. Addressing any of these issues could be a success for the organization. Even narrowly focusing on job-related outcomes presents challenges. The practitioners asked what constitutes a good wage? How long should someone stay at a job for it to be considered a good placement?

And many organizations want to go beyond preparing people for their next job. They want to set people up for lifelong success. “What does it mean to be educated?” asks Sarah Riley, Executive Director of High Rocks, a grassroots community nonprofit organization which provides educational programs for young people in the Central Appalachian region. “Our answer to that, after asking that question over and over again and talking about it and reflecting on it, is that what it means to be educated is to be an incredibly skilled, flexible, and strong learner.”

So, defining and measuring success can be a challenge. On top of that, it’s not always clear which successes can be attributed to a program or simply to a motivated individual.

The organizations represented at the event manage this challenge by monitoring their own work and collaborating with the people they serve. Sarah Riley describes a process of “co-creating with participants” by tuning into “what they need, what they're asking, what you're seeing, what you're learning, and doing that in partnership together.”

This is a labor-intensive process which doesn’t always generalize between organizations. To scale and share lessons, we need data and data analysis. Speaking to the workforce development leaders, Hanson emphasized this point: “That’s what data analysis allows you to do. It allows you to tell stories. It allows you to fixate in the minds of mayors and governors and foundation presidents why what you do matters and why it deserves support.”

The participants have already begun building their data. They collect data on their own programs, and they are working with the states they serve to obtain detailed government data.

Reimagining the Economy plans to help here: unifying data from multiple sources, providing and analyzing data, and bringing cutting edge tools to the problem. For example, Hanson suggested using AI to analyze the text of client interviews as a way of systematizing the information organizations are already gathering.

Many questions remain, but there’s no doubt good work is being done. SOAR, one of the host organizations, has helped thousands of people get job ready and find work and secured funding and training opportunities for hundreds of startups. Colby Hall, Executive Director of SOAR, is optimistic. “It's like the stock market,” he says, “it has its ups and downs, but the general trend is up.”