Capstone Project: Pandemic Professors

A group-based Master's Capstone project. Working alongside Pandemic Professors, a non-profit online tutoring service, to improve their tutor-student pairing process.

Overview
Project Goals
From January to August of 2021 I worked in a 5-person Capstone group on a project dedicated to improving the pairing process for Pandemic Professors, a non-profit tutoring organization. Pandemic Professors specializes in providing tutors to low-income students who had been adversely affected by the pandemic and distance learning. I served as the research lead for our team as we explored this space and developed solutions to help improve and automate the process of pairing tutors and students within the organization.
Our Client
Pandemic Professors is a non-profit organization run by a team of dedicated volunteers and host to hundreds of volunteer tutors. Their mission to serve low-income students continues to be highly important as these students grapple with continued adoption of distance learning and the fallouts of the pandemic. Pandemic Professors has very limited resources in terms of a development team, pairing staff, and other critical infrastructure. As such, they required a solution that was cheap, flexible and immediately implementable.
Pandemic Professors Logo
The Group
Images of all the team members
I worked alongside four other MHCI students for the duration of the project. I primarily worked as a researcher, but we supported each other in our domains of expertise and worked collaboratively across all facets of the project.
Initial research
Understanding the Problem Space
Screenshot of the an affinity diagramming session in Miro after a round of interviews
We began research into this space by conducting many long-form contextual inquiries within the Pandemic Professors organization. We spoke with department heads and volunteer pairers to understand how they conduct their day-to-day activity and what pain points they faced.

Additionally, our group spoke with parents, tutors and students to hear things from their perspectives as well. We hoped to understand the problems that parents faced while signing up for the service and the problems they had with tutors in the past. Additionally, we asked tutors and students for their experiences in sessions and paid particular attention to those who had been in previous pairings that fell apart. Within these interviews we also included some related stakeholders, such as school district employees and social workers who recommend the students for tutoring.
Defining the Pain Points
It was clear that many elements of the current pairing process and the related systems within Pandemic Professors were creating a strain for many of the stakeholders involved. Overworked volunteers within the organization spent hours pouring over disconnected spreadsheets attempting to match up tutors and students based on their availability and subject needs by hand. Tutors were breaking off relationships with students at higher-than expected rates, and re-matches with students created long delays in the system. We narrowed down a few key issues to improve with our prototype design:
Frustrated user on a computer

Manually comparing data like availability and subject needs put strain on the volunteer workers at Pandemic Professors. Comparing these values automatically would greatly improve the system’s efficiency

The current system does not evaluate tutors before initially pairing them. Our group worked to define a metric, called ‘Uncertainty Tolerance’, that we could repeatedly test open new applicants to determine how compatible they would be with the high-need students at Pandemic Professors.

One key issue that came up in several interviews was the lack of empathy between a tutor and student that would result in a dissatisfied tutor breaking off their pairing. We decided that a new integrated feedback system within the pair’s tutoring sessions could be created to help build empathy between the tutor and student.
Ideating a Solution
Areas of Focus
Our research efforts centered around designing, testing and iterating a suite of new tools for Pandemic Professors. We complete these design cycles in a series of two week ‘sprints’, becoming comfortable with the idea of rapid research and design. Each solution was conceptualized based on the pain points we learned from initial research and tested on target populations for further refinement.
Initial Storyboards created through Miro

We created a new set of tutor and student onboarding forms that standardized and synchronized sections across each form, allowing for automated comparison. For instance, language needs for students requiring a bilingual tutor were changed from an open-ended response to a series of multiple choice questions that could easily be compared across all tutors for each student to find them a tutor who matched their language request. These forms were tested on parents and the general population for usability via the think-aloud research method.

After speaking with empathy experts, our group determined that the best way to increase a tutor’s empathy for their student and extend the pair’s lifespan was not in a single onboarding training but rather through distributed practice of empathy and empathic listening. To achieve this, we created a new feedback form that used clinically-tested measures (ORS and SRS) to get tutors engaging in a dialogue with their students so that they could form a closer bond. These pointed conversations would be both about the tutoring sessions and about the student’s life outside of tutoring, building a holistic view of the student and allowing the tutor to empathize with them. We tested our form on a sample of student-tutor pairs, who incorporated it into their weekly tutoring sessions. We found that every tutor learned something new about their student and many actually committed to changing their tutoring style moving forward to reflect the feedback they heard from their student.

After interviewing tutors, parents and Pandemic Professors volunteers, we found that one consistent issue was that many tutors were unable to handle students with poor attendance or communication, and often requested to be re-matched away from those students. This was a recurring problem that led to the majority of re-matched in the Pandemic Professors system. We began referring to the tutor’s ability to deal with unreliable students as ‘Tutor Uncertainty Tolerance’. One of our main goals throughout the project was to find a way to measure this uncertainty tolerance as tutors first entered the Pandemic Professors system, so that pairing staff could avoid pairing intolerant tutors with low reliability students.

We developed a standardized measure called the Uncertainty Tolerance Assessment that Pandemic Professors could administer to new tutor applicants. This short assessment used scenarios adapted from our interviews to simulate highly uncertain situations. Tutor applicants have to choose between three options to progress throughout the story. Choices which reflect a greater degree of empathy or proactiveness earn the tutor more points. We coupled this multiple choice assessment with a short written portion where the tutor explains their rationale and describes how they feel about the outcome of the scenario.

We tested our scenario with over 100 participants, half of whom were general population participants that fit the criteria to become a Pandemic Professors tutor, and half of whom were active Pandemic Professors tutors. We found that there was a significant correlation between the scenarios in the test, meaning we were measuring a consistent metric across tests. Further, we discover that the more experienced a tutor was, the more they relied on established guidelines to direct their answers. As we wanted this assessment to be more of a litmus test for the tutor’s internal tolerance, we decided to only have new tutor applicants take the test, removing the experience bias that established tutors displayed. Finally, we found that the written responses across all populations were incredibly rich and helped build a better picture for how applicants valued our desirable traits like empathy and proactiveness. This led us to focus on developing a method for quickly evaluating the written responses, giving us a second metric alongside the multiple choice score.

Our team wanted to develop a functional database that would collect the data from these newly-created forms and consolidate it in one place. One pain point from the current system at Pandemic Professors was a lack of centralized data storage. This meant it was often difficult or impossible to track down information on a tutor’s past pairings. By creating this data center we wanted to provide them a faster way not only to create new pairs, but to monitor existing pairs and access data on older pairs. We iterated the data center multiple times by designing the functional prototype in Sheets before conducting think-aloud sessions with Pandemic Professors staff members.
A Functional Prototype
Lofi prototype Pairing Page
An early sketch of the Pairing Tab
Lofi prototype Pair Monitoring Page
An early sketch of the Pair Monitoring Tab
Lofi prototype Tutor Applicant Page
An early sketch of the Tutor Applicant Tab
One of our primary concerns in creating our prototype was that it could be immediately implemented by Pandemic Professors, given their lack of developmental resources. To achieve this we used free groupware within the google suite, including Forms and Sheets. Our functional data center could perform many different functions that assisted with Pandemic Professors’ day-to-day activity. Among its functions, the data center can

  • Track all new tutor applicants
  • Automatically generate pair recommendations between tutors and students
  • Collect all feedback form data on a pair
  • Alert Pandemic Professors when a pair is struggling based on feedback form data
  • Maintain a repository of all current and past pairs, including historical information on why a tutor or student’s past pairing was disbanded.
Additional Deliverables
Along with this data center we provided our suite of new forms that feed seamlessly into the data center and incorporate the core principle of generating empathy between the tutor and student.

Finally, we wrote a comprehensive manual to accompany our forms and data center so that Pandemic Professors staff could learn how to use these tools, as well as troubleshoot them in the future. The manual further explains our rationale in design choices throughout the prototype and provides links to the various forms and sheets themselves for easy reference.
Example pages from our handoff Manual

To read the entire manual, click here
A Vision for the Future
While we wanted to provide Pandemic Professors with a solution that would work now, we also wanted to give them a vision for a future state that brought their workflow out from google’s free groupware and into a fully-customizable platform which would expand both the functionality of the system as well as the experience of the user. We based our designs for this conceptual prototype (dubbed ‘Data Center 2.0’) on the capability and aesthetic of an application built in Flutter, Google’s free UI toolkit, to reflect how affordable and realistic an implementation like this would be with a small development team devoted to it.
The 2.0 prototype was developed in Figma and used a series of tabs as its core navigational framework, much like the original Sheets-based prototype. Unlike the functional prototype, we used many of our core findings from interviewing Pandemic Professors team members to focus on the handoff experience between various groups within the organization. The new prototype could cetner itself around tasks depending on the department and clearance level of certain staff members. Interactions between staff were emphasized in the prototype as well, with built-in prompts for generating messages as one staff member delivers a pair to another for approval.
The Final Package
Lofi prototype Pairing Page
Lofi prototype Pair Monitoring Page
Images from our 80+ page research report
Lofi prototype Tutor Applicant Page
We presented our research and prototype to the client, Pandemic Professors, in August of 2021. This included an hour-long presentation, a research report and a project website. We held several meetings with the client at the end of the term where we discussed handoff details and walked them through using the prototype. Our project website can be accessed here:

Link to the Project Website
Created by Brady Baldwin, 2021