Assessing Sagence Learning's First Adaptive Math Supplement

Overview

In February of 2016,
Sagence Learning, Inc. launched its first adaptive Math Supplement, designed as a refresher
for the prerequisite math skills necessary for success in higher education and
the workforce. Sagence Learning partnered with SUNY’s Brooklyn Educational
Opportunity Center (not a college or university, but an academic and workforce
development enterprise) to deliver this supplement to four pre-algebra classes
over the course of two school quarters. Sagence Learning worked with author and
Professor of Mathematics John Redden to design and develop the content and instructional
strategies for this offering. The supplement included three objectives
(operations on real numbers, fundamentals of algebra, and intermediate topics
of algebra) presented as adaptive lessons with targeted questions and related
activities (readings, videos, and practice exercises) to help students refresh
their knowledge and skills. The below findings and recommendations are based on
the experiences of two faculty members and 62 student participants, gathered
through focus groups and surveys as well as behavioral analytics collected by
the platform.

‍

Key Findings

The points below
summarize key findings of the pilot evaluation, which includes answers to
defined research questions and other insights that emerged from the study. 

· 78 students were
enrolled across four classes; of those, 62 students actually logged time in the
platform. The average amount of time students spent working within the supplement
was 5.9 hours. The program was designed to take students approximately 15 hours
to complete, if (besides completing all quiz questions) students accessed and
engaged with the recommended learning activities.

· 13 of those 62
students made multiple attempts on their formative quizzes. These students
improved an average of 6.5% from their first attempt to their last attempt.

· Students affirmed
that the quizzes were an appropriate way to measure their own progress and
enjoyed the ability to assess their own knowledge as many times as needed.

· Faculty
consistently stated that they did not have sufficient time to properly leverage
the Math Supplement given the time restrictions and amount of content they were
required by BEOC to teach.

· Of the seven
students who were surveyed, all felt they were able to be self-directed when
completing the personalized lessons, five agreed that the Math Supplement
helped them to better understand the algebraic topics and six felt the
supplement prepared them for the next step in their academic career.

· Within the focus
group, students also responded favorably to the learning journal, pace and
progress bar, quizzes and overall layout of the content. 

· Not one student
finished all of the formative quizzes or completed the supplement. Two students
were able to finish up to 60% of the formative quizzes. Student focus group
participants commented that they did not have the time to complete all the
quizzes and use all of the content. Many were taking class in the evenings
after a full day of work.

· Faculty expressed
satisfaction at the content quality of the supplement and the ability for
students to be self-directed.

‍

The Future of Supplemental Courseware at Sagence Learning

The
recommendations presented below are based on the results collected in the pilot.
They aim to support Sagence Learning’s mission of providing innovative, alternative
and flexible approaches to higher education that connect individuals’ unique
and diverse lives to their personal learning goals.

· Partner with
faculty to understand needs of the classroom environment and how the supplement
can be properly leveraged. By customizing a supplement/competency to an
institution/program's needs, it will increase faculty's motivation to leverage
the offering, and make the offering more useful for students in their
coursework. Sagence Learning could also explore providing the tools to
faculty/administrators to customize content themselves.

· Integrate more interactivity
to provide students who are struggling with varying opportunities for practice
and learning (not just multiple choice questions). This could be in the form of
a guided tutorial, fill-in, click and reveal, additional hints within quizzes,
etc.

· As part of the
completion of the class, require students to complete the supplement,
potentially giving it a weighted grade, based on how much of the supplement
they have completed.

· Any training on the
platform to external partners should be based on value-add to faculty and
students. The different users should better understand how the platform could
better leverage their roles.

· Consider expansion
to courses or programs that strategically leverage the personalized lesson
format, or use as proof of prior learning.  

‍

Conclusion

The Math Supplement
pilot requires more of a partnership with a learning institution and planned execution
for best usage. Students are attracted to the self-directed nature of this
format, and capability to consistently test themselves on algebraic topics. The
faculty and students found the Math Supplement valuable and students improved
in their academic knowledge as they progressed through the supplement.

‍

Acknowledgements

This case study was written by Shawn Gunaratne, Evaluation Specialist at Sagence Learning, Inc., and benefitted from interviews and correspondence with the faculty and students at SUNY Empire State College's Brooklyn Educational Opportunity Center.