Introduction

The immediate employment of those facing the highest barriers to work is often not the only goal that social enterprises have. For most, the mission is to help their employees find long-term success in the workforce. While the impact a social enterprise is setting out to achieve may vary depending on organizational goals, the measurement of employee outcomes poses several challenges to social enterprises. Collecting data on employees many months, or even years, after their employment at the social enterprise is particularly challenging. And this only gets harder with the more time that has passed. However, it is precisely that data which is most valuable. Social enterprises need to learn the long term effects of social enterprise employment in order to make programmatic improvements and to demonstrate to others the efficacy of their intervention.

This learning guide, the second in a series on impact measurement for social enterprises, will explore the ways to approach identifying what to measure and how to measure it.

 

The challenge

Let’s take two fictional examples of café social enterprises – Café Rouge and Café Bleu. Management of each wants to know how successful their social enterprise is at helping people retain work in the long term. Staff at each have attempted to follow-up with 100 employees who worked at the café and exited approximately one year ago. They want to learn how many of their employees are employed elsewhere one year after their time with the café ended.

What are the retention rates for each? Is Café Bleu’s retention 40% (40 out of 100 people) or is it 80% (40 out of 50 people)? Is Café Rouge’s retention 30% (30 out of 100 people) or is it 100% (30 out of 30 people)? There’s not necessarily a “right” answer to this question, and neither is perfect. Ultimately, it is a call on what the appropriate denominator should be – the total participants or total participants you were able to reach.

The industry norm is that denominator is the number reached. This is because it can be considered statistically sampling and is representative of the people not reached, since there are both positive and negative reasons for being unreachable. However, this is true only  if you attempt to reach all participants, not just positive exits.

The most important thing is to make sure that you are consistent and clear about how the calculation is made. In order to do so, REDF recommends this three step process when defining and collecting program impact data:

1. Define what you mean by “employment” at the social enterprise
2. Define what you mean by “retention” and what other metrics you want to collect
3. Set a schedule and implement

 

Step 1 – Define “employment”

In order to measure the effect of social enterprise employment on an individual, we must first define what we actually mean by “employment” at the social enterprise. As with many things related to social enterprise, this is more complicated and nuanced than it may seem. For example, if someone fills out a W2 but quits on their first day, are they considered as having been employed?

Let’s take a fictional example of an employment social enterprise. On average:

• 200 people show up for the quarterly information session. Basic contact and eligibility info is gathered.
• 100 people come back for day one of the two-week training program. Full intake data is gathered.
• 50 people successfully complete the training program (meaning they pass attendance and proficiency requirements) and interview for a job in the social enterprise.
• 45 people accept an employment offer from the social enterprise.
• 30 people show up for their first day of employment.
• 15 people show up for day two of their employment.
• The majority of people who show up for day two of employment are still employed at the social enterprise 30 days later.

For this social enterprise, at what point would you consider someone as having been “employed”?

While there is not a right answer to this question, it is essential that your social enterprise has an answer for itself, and one that is consistent over time. To do so, you must determine a threshold at which someone is considered as having been employed at the social enterprise. For example, the threshold for employment could be showing up for the first day of employment – 30 people in the above example. Or it could be showing up for the second day of employment – 15 people in the above example.

If you err on the side of more broadly defining employment, you might be understating your outcomes by including a segment of people who are not yet ready for your intervention. On the other hand, if your threshold is too high, you might be overstating results by only reflecting outcomes for people who have successfully ‘attached’ to your program. Be aware of missed opportunities for improvement below the threshold.

It is a good idea to speak to other workforce providers and your local workforce board before solidifying your definitions. This will make sure that your definitions are roughly in line with other programs in your area and/or meeting the requirements of your workforce board, if applicable.

 

Step 2  – Define long-term metrics

Just as it is essential that we clearly define employment, it is equally important to clearly define the metrics by which we determine success. To understand why, let’s take a look at two fictional social enterprise employees, Jake and Chris:

As you can see, Jake and Chris have had significantly different experiences after they left the social enterprise. Chris has been employed continuously at the same job for the entire time since leaving the social enterprise, whereas Jake has been employed at three different jobs with periods of unemployment in between.

However, if we were to follow up with both Jake and Chris one year after they have left the social enterprise and ask them “are you currently employed?” we’d get the answer of “yes” from both of them. This is not inaccurate, per se, but we are losing the opportunity to collect a more nuanced picture of our employees’ post-social enterprise experience.

Subsequently, it is recommended that your social enterprise defines retention as clearly and specifically as possible, and collects metrics that capture it accurately. Instead of asking “are you currently employed?” perhaps ask “have you been continuously employed for 12 months?” Again, there is no universally correct way to do this, only the right way for your social enterprise. What matters most is consistency.

In REDF’s experience with our own grantees, as well as with broader workforce field practice, long-term employment retention can be measured in many different ways. Common metrics include:

• Are you currently employed? Were you working in the last week/month?
• What percentage of the last year were you working?
• In the last year were you ever out of work for more than one month at a time? Have you been continuously employed since we last spoke?
• Have you been in the same job since we last spoke?
• Have you been continuously employed in a living wage job?

Asking these questions will get your social enterprise far more information on the person’s work experience after their time with you. You will be able to understand how long people are staying in certain jobs, how many times they are switching jobs, and any time spent not working. This information, in turn, will allow you think about ways to adjust your programming both during and after the social enterprise experience accordingly to better meet your goals for your participants.

It is also worth keeping in mind the behaviors your metrics might be unintentionally incentivizing. For example, if you are asking whether the employee has retained the same job continuously for 12 months, you might be incentivizing employees not to take new job opportunities in their first year, regardless of whether it is appropriate for them to do so.

Also, be sure to take full advantage of having contacted someone. This is a great opportunity for other data collection about your social enterprise and employee outcomes. Other data that are commonly collected include:

• Education/credential attainment
• Wages
• Housing stability
• Substance use
• Employer-provided benefits
• Job structure/convenience
• Attitudes and beliefs about work and self
• Reconnections with family
• Elimination of fees/ garnishments (court, DMV, child support)

 

Step 3 – Set a schedule and implement

Now that you have determined who counts as having been employed by the social enterprise (step one) and what data you want to collect (step two), you can begin effectively tracking workers after their exit from social enterprise. It is best practice to follow up with all people who meet the defined employment threshold, regardless of their reason for exit, for a least 12 months post-exit. Be sure to track not just the number of people you successfully reached, but also the number you attempted to reach.

While successfully reaching 100% of your employees one year or more after they have left employment is difficult, there are a number of strategies you can adopt to increase the likelihood of reaching someone for follow-up:

Plan ahead – Accuracy and consistency are crucial for good data collection, so ensuring that data collection is done at routine intervals is essential. Be sure to create a schedule and stick to it. Common intervals are 3, 6, 9, and 12 months after exit. Attempts every month can be burdensome, while waiting longer than 3 months risks waiting too long to get in touch with someone after they leave.

Communicate – Before they leave your social enterprise, let your employees know when and why you will be contacting them in the upcoming months or years. It is also a best practice to remind your employees that if they change contact information, they should let your organization know so you can keep their records up-to-date.

Staff accordingly – This can be a time consuming process, so it is essential you staff accordingly and have clearly defined roles and responsibilities.

Incentivize responses – Many social enterprises incentivize the collection of data with gift cards and cash rewards for participants. Typically these values range anywhere from $5 to $100. This depends largely on the capacity of the social enterprise both in terms of number of employees as well as ability to pay for this data.

Leverage technology – Many different systems are used by social enterprises to collect and analyze data. Excel and Google Docs are popular among start-up social enterprises, and HMIS and Share Point popular among larger social enterprises. Increasingly popular is the cloud-based Salesforce platform. We’ve found that there is decreasing use of Microsoft Access and ETO. This is in addition to specialized industry software that is often used in tandem to manage businesses, such as TempWorks or other staffing software, retail and restaurant point of sale software, and e-commerce and inventory management platforms.

REDF has found that most organizations use at least two data systems, plus at least one funder-mandated system, and go through a major system change around every 7 years. Don’t waste time trying to figure out a single perfect system. You can streamline, but don’t get beaten by the pursuit of perfection.

 

Conclusion

Collecting and measuring impact data is an important part of running your social enterprise. Like the old saying goes, you can’t manage what you can’t measure. If you can’t measure how successful your employees are in finding long term success in the workforce, how can you make improvements to your programming to further help them? This data collection is important, even if your social enterprise has had or will be having a third-party evaluation. It provides unique information for the purpose of routine performance insights and program improvements, and it will help you communicate the value of your program to outside partners. But use it or lose it – don’t go through the effort of collecting data that you don’t ultimately use.

In our forthcoming learning guides on this topic we will explore the:

• Different types of formal evaluations and what you should consider before pursuing
• Data systems to facilitate impact measurement initiatives
• How to use impact data to approach funders