In Part 1 of our predictive analytics series we discussed the origins of the discipline, its evolution in higher education, and shared a foundational set of strategies from our experience in the field.
In Part 2, we explore the ethical implications of machine learning, how we use the outputs of our models, and our results to-date.
Inclusion vs. Exclusion
When it comes to how we use the generated predictions, our philosophy runs contrary to the norm. In the private sectors of many industries (such as healthcare and finance), predictive risk models are often used either in a predatory or exclusionary way—to take advantage of certain groups while refusing service to others. The predictions themselves may or may not be true, but are acted upon without the benefit of a human interaction. These companies are incentivized to identify the most profitable opportunities no matter the downstream effects.
At ReUp, we leverage our predictions with the goal of facilitating inclusion—never exclusion. We are able to ensure this because the value of inclusion is in our DNA; we are only compensated for our work when students re-enroll and as they progress through school. To that end, we use predictive analytics to customize our interactions with all students.
Here’s a practical example:
- We find that students with a high likelihood of returning to school are also typically more prepared to return and have fewer obstacles in their way. We’re able to prioritize them in our outreach for the closest academic term and place them on a faster support track with increased automation and lower human touch.
- Students with a medium likelihood of returning are a special group, as we’ve found that this is where the human side of our coaching services can make all the difference in empowering a student to return and equipping them with the tools to succeed once back.
- And for students with a low likelihood of returning, we find they typically have more fundamental barriers in their way and the lowest overall readiness to return, so we set them on a longer journey to better understand what they value and what they may need before taking the next step.
It is through this inclusionary and personalized approach that we’re able to support every student, while optimizing our service as a whole.
Over time vs. One-time
Timing can be hugely consequential when predicting human behavior. Each person has a unique set of factors at play in their lives that evolve and change over time, so when predictive models are only applied at a single point in time they typically underperform.
To account for this, ReUp’s platform incorporates time itself as a variable in our models so that the predictions we make are as dynamic as people’s lives. This means that each day, all the students we serve receive a new set of predictions based on the latest data we have available. This ensures that the actions we take to support students are up-to-date, making them far more accurate, useful, and effective.
Tuning for Equity
When working with underserved or at-risk populations, it becomes of highest importance to invest the time and energy requisite to understand the ethical ramifications of technology usage in service delivery. In ReUp’s case, we treat this responsibility with the utmost care by ensuring that every model we develop is assessed for evidence of bias or discrimination and tuned accordingly.
Our guiding principle here is to accelerate equity, not replicate inequity. Nationally we know that 30.8 percent of black adults and 22.6 percent of Latino adults have earned an associate degree or more, compared to 47.1 percent of white adults between the ages of 25 and 64, according to recent reports. These known equity gaps lead us to favor algorithms that are highly interpretable, rather than leveraging black-box methods that may be highly accurate but can’t be fully explained. Each N-of-1 matters and we are unwilling to sacrifice our understanding of the individual for the potential of a marginal gain in performance.
(And our entire platform is both NIST and FERPA compliant to ensure that all student data is protected and secure.)
Proof in the Pudding
Through spring of 2019, ReUp helped over 8,500 students return to school and recaptured $25M in tuition revenue for our partners. Our partnerships have grown from 3 to 43 in the last 18 months, and we are currently working with over 150,000 students—projected to surpass a quarter million by the end of 2019. And as a direct byproduct of developing our technology platform, we doubled the effectiveness of our coaching team while simultaneously quadrupling our efficiency.
As we look toward the future, we will test and refine our platform, predict additional behaviors (like melt), and dive deeper into the world of personalization. Through both innovation and iteration we hope to further unify the art of coaching with data science. Together, we believe we can create a more dynamic and equitable ecosystem in which all learners have access to the supports they need to pursue their goals and thrive.
___________________________________________
We hope this discussion has helped to illuminate some of the ways we are leveraging data science to improve student outcomes in higher education. If you have thoughts, questions, or feedback, we’d love to hear from you!
