Brandman University launched a competency-based direct assessment program. Designers of the BBA program focused their attention on ensuring flexibility for the student through use of technology, offering high quality online content, and incorporating faculty expertise into design process.
Leverage our complete white label degree program and course catalog to get online fast.Strut’s customizable course catalog provides a rapid (30 days) pathway to online courses with zero capital expenditures.Schedule a call now.
This case study is part of a series on competency-based
degree programs that have been emerging in recent years. The case studies are prepared by the Council
for Adult and Experiential Learning (CAEL) with funding from Lumina Foundation.
In October 2014,
Brandman University launched a competency-based direct assessment program. Brandman University is a private, adult
focused, non-profit institution based in California. The competency-based Bachelor of Business
Administration (BBA) is a low-cost and flexible alternative for adult and
nontraditional students. This competency
model draws on three educational frameworks: the Lumina Degree Qualifications
Profile, the Association of American Colleges & Universities (AAC&U)
Essential Learning Outcomes, and industry standards as detailed by the
Occupation Information Network (O*NET).
Designers of the BBA program focused their attention on ensuring
flexibility for the student through use of technology, offering high quality
online content, and incorporating faculty expertise into design process.
The Brandman BBA degree takes place entirely
online; it integrates text and video components, formative assessments within
each competency, and faculty and academic coach interaction. Brandman developed a catalogue of over 80
competencies for the degree program.
Students choose one of four specializations: information systems management,
supply chain management and logistics, management and organizational
leadership, or marketing.
Depending on the specialization, the student must complete between 56 and 61 of the 80-plus competencies to earn a degree.
Each competency is evaluated through an objective assessment that could take the form of an auto-scored test, student project, or portfolio, as appropriate to the specific competency. After graduation, which is expected to take about to and a half years for students with a previous degree or four years of study for students with no previous college credit, the demonstrated competencies are represented in dual transcripts that show both the competencies and the traditional course equivalents.
Brandman University, part of the Chapman University
System, is an institution with 26 campuses throughout the states of California
and Washington and includes robust online offerings. The institution serves nearly 12,000 students
per year. A majority of Brandman
students enroll with considerable transfer credit, and a primary focus of the
university is to meet the needs of adult learners in dynamic and flexible ways.
Brandman first began
looking at ways of incorporating competency-based education into its programs
in 2011, after the release of Lumina’s Degree Qualification Profile (DQP). For its first endeavor with competency-based
education, Brandman chose to revise and refine its general education
requirements based on competencies drawn from not only the DQP but also the
AAC&U’s Liberal Education and America’s Promise (LEAP) Essential Learning
Outcomes as well as from the institution’s own mission and values
(Klein-Collins, 2012). Brandman students
are required to demonstrate competency in several areas; broad, integrative
knowledge; applied learning; innovation and creativity; civic engagement; global
cultures and integrated learning. Students
demonstrate competency through assessments that are embedded in all
Brandman courses, with grading based on the AAC&U’s Valid Assessment of Learning in Undergraduate Education (VALUE)
rubrics. This adoption of a competency
framework provided a foundation for building a
wholly competency-based degree program
at Brandman.
In 2012, Brandman administrators
surveyed the landscape of higher education and
concluded that the time was right to become involved
in a major change initiative.
Competency-based education had
recently surfaced as a model with
growing momentum, and
Brandman leadership determined that CBE was a great way to respond to changing student
needs. They believed that a
CBE model would appeal
to
learners who are interested in moving at their own pace, having
a subscription for unlimited
access to academic
materials rather than paying per credit, and using a variety of technologies-
tablets, mobile applications, computers, etc.- in order to fulfill
requirements.
As a
first step in the development of a CBE degree, administrators at Brandman began
by investigating what students themselves wanted and needed. The institution surveyed approximately 1,00
prospective students on the following five areas: 10 the viability of a CBE
degree program; 2) the target market for CBE- millennials, newbies, adult
learners; 3) the most suitable degree programs for CBE; 4) the preferred
business model for CBE- accreditation, financial aid, self-service, online
tutoring, liberal options for transfer of credit; and 50 the price that the
market expects to pay. Based on the
feedback from this survey, Brandman administrators and faculty began working on
a competency-based bachelor’s degree in business administration with options
for specialization.
From the earlier DQP initiative, the program designers already
had
considerable expertise in defining the essential competencies of a Brandman
graduate and incorporating
competency-based assessment approaches into degree programs. Therefore, when faced with
the
task of constructing an entire degree based solely on competencies, Brandman chose to lean heavily on its own internal expertise.
According to Laurie
Dodge, vice chancellor of institutional assessment and planning, many institutions
that develop CBE programs choose
to follow
a deconstruction/reconstruction model: begin with an existing program, identify its learning outcomes, and reconstruct the program in a way that reflects the outcomes of the original. In contrast, Brandman created the Bachelor
of Business Administration from the ground up, or with what Dodge calls a “framework approach,” starting with the
end result first by asking questions like “What should students
be
able to do, know, and understand upon graduation?” The designers built the degree by integrating competencies and outcomes from three accepted frameworks:
1.
The Lumina Degree Qualifications Profile, which describes what students at the bachelor’s level need to know and be able to do:
a.
Knowledge (broad, integrative knowledge and specialized knowledge)
b.
Intellectual Skills
c.
Civic Learning
d.
Applied Learning
2.
The Essential Learning Outcomes
articulated by the Association of American Colleges & Universities’ Liberal Education and America’s Promise initiative:
a.
Knowledge of Human Cultures and the Physical and
Natural World
b.
Intellectual and Practical Skills
c.
Personal and Social Responsibility
d.
Integrative and Applied Learning
3.
Industry standards as
detailed
by certain industry-specific
certifications and
by the Occupation Information
Network, developed by
the U.S. Department of Labor Employment and Training Administration
The DQP and LEAP frameworks contributed to the building of a robust set of general education competencies to be embedded across all levels of the new competency-based degree
program. First, program designers used these 2 frameworks to
identify
13 general education competencies that would be required for the BBA degree. Then, for the major-specific
competencies and the specialization
requirements,
program designers consulted a business
advisory council;
the standards of industry-specific
certifications, such as the American
Medical Association; as well as the publicly available database O*NET, which provides hundreds of occupational definitions and learning outcomes associated with specific jobs.
Informed by these
three frameworks, faculty identified
competencies in the form of outcome statements, which are robust statements of what the students should learn to do for each of the major
academic areas of the degree. In order to ensure that the new program structure
would not be a simple reconstruction of a traditional business
program, administrators urged the faculty to avoid consulting the Brandman
course catalogues and instead focus on relevant discipline specific outcomes.
During this process, the faculty from multiple disciplines worked together
to ensure that the
larger
University Degree
Qualifications were embedded within each step of the program. Throughout the development process,
the business advisory council provided feedback from the employer’s perspective, and the faculty
members provided feedback from
the university’s perspective. This
process produced a list of over 80 unique competencies that are part of the
overall BBA degree framework, divided among the three categories of general
education, business core, and area of emphasis.
The general education competencies are
comprised of 13 competencies in 6 subdomains.
The business core has 34 competencies in 7 subdomains. Each of the areas of emphasis has its own
subdomains and competencies. Examples of
each type of competency are provided in Table 1.
The complete set of competencies
for the BBA degree
can be found in the online Brandman catalogue at: https://www.brandman.edu/files/attachments/2014-2015-Catalog.pdf
In order to complete a degree, students must demonstrate 13
general education competencies, 34 business competencies- referred to as the
business core- and between 9 and 14 competencies in an area of emphasis (Table
2).
From the competency framework created by Brandman
faculty, a completely new curriculum was designed that would successfully guide
students through learning activities that would develop each of the 80-plus
identified competencies. The trajectory
of the program is such that the competencies are scaffolded, meaning that higher-level
competencies cannot be accessed until a perquisite competency is first
mastered. In this way, students build on
their skills as they progress (see Figure 1).
The scaffolding of the Brandman
BBA modules
is seen not only in the
bundle sequencing but also in the assessment process. Each module has both a cumulative/summative
assessment at the end as
well as formative assessments through- out.
Summative assessments occur at the end of each module, and successful completion of a summative assessment signals the demonstration of a competency. Students also
engage in formative assessments throughout
their learning journey within each competency, completed
through the use of an adaptive learning engine,
which generates custom designed learning
activities and subject matter content based on
what students know and do not know. The formative
assessments for all competencies are
brief, objective-based assessments linked to the
adaptive learning engine. These assessments
not only test student progress within the module
but also require students to demonstrate
confidence in their answers. Students must both
know the correct answers and be confident that these answers are correct.
The confidence index
assists students in building trust in their learning,
and it provides data for the adaptive learning
engine. If a student does not pass a formative
assessment, the adaptive learning engine will
send students back to the content that they
need to spend more time on in order to be successful
the second time around. The formative
assessments can be taken as many times as a
student needs to in order to successfully progress
through the module.
“Assessment, or student demonstration of
mastery, is at the heart of competency-based
education,” says Dodge. In designing Brandman’s
approach to assessment, the faculty team recognized
that the format of each assessment needed
to be appropriate for the learning outcomes and
competencies. This is why Brandman employed
a backward design, meaning that instead of basing
the assessments on the learning activities, as
in a traditional program, they opted to create
the assessments based on the intended learning
outcomes and then constructed the actual
learning activities as foundational elements
for the assessments. After viewing Brandman’s course outcomes through Bloom’s Taxonomy,
which is a holistic system of classifying learning
objectives in which learning at higher levels is
dependent on first gaining knowledge at lower
levels (Krathwohl, 2002), it became evident to
the faculty early on that the competency assessments
should be categorized into two separate
levels (see Figure3).
Level 1 competencies involve learning that is
primarily focused on the understanding of information
and concepts, processes which are found
in the lower levels of Bloom’s Taxonomy, particularly
Remembering and Understanding (Figure
1). Level 1 competencies, which make up about
40% of the competencies for Brandman’s BBA
degree, are best assessed through an objective based
final assessment through which students
can demonstrate recollection and understanding
of course material. Of the 13 general education
competencies, 3 are level 1; and 18 of the 34
business core competencies are level 1.
Level 2 competencies pertain to the application
of learned concepts and skills as well as higher level
analysis, which requires the integration of knowledge.
These processes are found in the higher levels of Bloom’s Taxonomy, such as Applying and
Analyzing.
Level 2 competencies constitute about
60% of the total competencies for the BBA and
are assessed through performance-based projects,
such as presentations, portfolios, or reports, which
are graded using a rubric developed by the faculty
and an in-house psychometrician. Of the 13 general
education competencies, 10 are level 2; and 16
of the 34 business core competencies are level 2.
In addition, nearly all specialization competencies
are level 2.
From the early stages of the development of
the competency-based BBA program, Brandman
administrators planned for the program to be
technology-driven and, therefore, easily accessible
to working adult students. To accomplish this,
Brandman partnered with Flat World Education,
a digital publishing company that specializes in
education and digital content.
Flat World entered the scene to help create
an online platform that was easy for students to
navigate as well as to assist in the development of
the online content. Since CBE was a new field for
Flat World, Brandman worked hand-in-hand with
them to create a customized and original platform
that could hypothetically be used by other
institutions interested in following a similar path.
To make sure the platform would meet student
needs and expectations, Flat World independently
conducted research on prospective users while both Brandman and Flat World conducted
joint research with the first group of CBE students.
The cooperative research effort focused
on how information would best be presented to
CBE students, as well as how they best interact
with an online educational platform.
The resulting product of the partnership with
Flat World was, according to Dodge, a “highly
adaptive digital space” that students can access
from a variety of devices. Tablets like the iPad
are the most popular device, but many students
also use laptops, and often both. Over 30,000
pages of content, much of it original, are available
to students through the platform. This
content is dynamic and can be edited or restructured
if data analysis demonstrates that a change
could improve student outcomes. Students have
access to embedded text with videos integrated
into the narrative and can participate in discussion
boards and video-conferencing.
Since it is so difficult to predict the needs or
usage patterns of students in a brand new program,
Flat World provides real-time updates and
reports on the operations of the platform as well
as student interactions with the embedded text.
This way, changes can be made quickly if and
when problems are identified.
The BBA program is offered as a subscription
model, with students enrolling at any time for six-month terms. Students can start a term on
any Monday during the year. The subscription
model allows students to work at their own pace
to complete as many modules or bundles as they
can during the six-month period. Dodge expects
that due to the self-paced nature of the program,
the time to complete a degree may vary widely
but the typical enrollee will be able to complete
an undergraduate degree within 30 months, or
about two and a half years.
Tuition for the BBA is a flat, per-term fee
of $2,700, which includes digital textbooks.
Students completing the degree on schedule—
within the expected two and a half years—would,
therefore, expect to pay a total of $13,500 for the
bachelor’s degree. A student going at a slower
pace, and completing the degree in four years,
would pay $21,600.
The program designers recognized that the
competency-based BBA program at Brandman
may be ideal for many students yet might present
certain challenges for other students along
the way. Therefore, it was important to make sure
students have the support they need to succeed.
The program designers addressed this concern by
providing guidance to the students both before
enrollment and throughout the program.
Before a student enrolls in the competencybased
BBA, Brandman requires them to complete
the Competency Intro Module (CIM),
which introduces the technology and how to
navigate it, what to expect from competencybased
coursework, and how to fit education into
the students’ otherwise busy lives. Students who
do not complete the three week CIM successfully
are not admitted into the program and are advised that they may be better candidates for
traditional or blended online programs.
As noted earlier, faculty played a central role
in the development of Brandman’s competency- based
BBA program by identifying competencies,
working with specialists to build assessments,
and deciding the content of the various
learning activities. The transition into a competency-based
format also necessitated a shift in
the traditional roles and responsibilities of faculty
to ensure that students have the support they
need to be successful. With the launching of the
CBE program, Brandman University now has
two faculty models: one model for its traditional
credit-based programs and another model for
the self-paced competency-based programs. In
the traditional programs, full-time faculty teach
and support the credit-hour blended and online
programs. In the self-paced competency-based
programs, full-time tutorial faculty are responsible
for ensuring that students understand concepts
and content. The university’s reassembled
faculty model for the competency-based programs
has four separate faculty roles:
1. Curriculum Developers: Curriculum
developers are the subject matter experts
who design the competency-based
courses. These faculty members are primarily
full-time academics employed by
Brandman University who normally teach
and provide oversight for the credit-hour
blended and online programs offered by
the university.
2. Tutorial Faculty: Tutorial faculty are the
subject matter experts whose primary
responsibilities include tutoring students
in the university’s competency-based
programs, maintaining currency in his/her
discipline, and actively participating in the
university’s continuous improvement processes.
These faculty ensure that important
concepts are understood and that student
questions are answered.
3. Academic Coaches: Academic coaches are
the advisors who counsel students regarding
competency progress and program requirements. They manage student activity
reports and ensure students are regularly
participating in education activities.
4. Assessment Graders: Assessment graders
are the subject matter experts whose
primary responsibility is to accurately and
consistently score student submissions
of performance-based summative assessments.
They are responsible for providing
robust feedback on each scoring event.
In addition to the interaction and support of
the above staff, students in the competencybased
programs also have access to online communities
that can offer assistance when needed.
Brandman University has established online
writing and math communities that are open to
all students, not just CBE BBA students.
While the BBA offers students a competency-based
experience that is situated outside
traditional time measured agendas like credit
hours and 15-week terms, this innovative structure
means that the program does not meet
many of the credit-based regulatory requirements
for federal financial aid. Brandman, however,
successfully sought approval from the U.S.
Department of Education (DOE) to offer the
BBA as a direct assessment program (see box).
This status for the BBA program permits some
adjustments in the regulations to allow students
to use federal financial aid for the program.
The first step in the process leading up to DOE
approval was to have the program approved by
Brandman’s regional accreditor, the Western
Association of Schools and Colleges (WASC).
Because the BBA program differed significantly
from Brandman’s credit-based and course-based
offerings, the approval process necessitated a
site visit for what is called a substantive change
review, conducted by peers. In this thorough
review of the program, WASC focused its attention
on the details of the student’s educational
journey as well as credit hour equivalencies.
In particular, WASC required assurance that adequate academic support would be provided,
that there would be appropriate use of technology,
and that assessments would be rigorous.
After a satisfactory site visit, the Structural
Change Committee (a standing committee of
commissioners) recommended to the WASC
commission that the program be approved. This
entire process, from notification of intent to final
approval, took around 11 months.
According to Laurie Dodge, the process of
seeking accreditation through WASC ensured
that every aspect of the BBA program was well
thought out and intentional. In addition, the
thorough review conducted by WASC, and their
interest in the direct assessment nature of the
program, helped Brandman find the best way to
describe the program when seeking approval
from the U.S. Department of Education.
After the BBA program was approved by
WASC, the next step in seeking direct assessment
approval from the DOE was to apply to
the Secretary of Education to have that program
determined to be eligible for Title IV. This process
required the submission of an application
addressing how the CBE program meets a set
of criteria outlined in a March 19, 2013, Dear
Colleague letter. Institutions applying for direct
assessment must explain its methods for “equating
the direct assessment program to credit
or clock hours and related parameters of the
program, including minimum weeks of instructional
time, payment period, how an academic
activity will take place during each week, and the definition of a full-time student” as well as
how students “will interact with a faculty member
on a regular and substantive basis.” The
institution must also “address issues such as
how it plans to measure satisfactory academic
progress (SAP) for students in the direct assessment
program” (U.S. Department of Education,
2013). Once the application was reviewed, the
DOE posed a series of additional questions to
which Brandman replied in writing. At the end of
this process, the application was accepted by a
DOE committee, and Brandman’s BBA program
was approved for direct assessment in October
2014, five months after application submission.
Direct assessment programs are not exempt
from federal financial aid regulations; they merely
have additional flexibility in meeting those regulations.
There are two regulations that are particularly
challenging for self-paced CBE programs: 1)
those related to the student’s “satisfactory academic
progress” and 2) those related to “substantive”
interaction with faculty.
In a traditional credit-based program, financial
aid is disbursed term by term, and uninterrupted
disbursement of financial aid is dependent upon
the student making satisfactory academic progress,
or SAP, which means successful earning of
credit hours toward the degree within a given
time period, such as a term. In the case of programs
like the Brandman BBA program, there
is no standard number of competencies that a
student is required to earn during any one six month
subscription period. For the purposes of
federal financial aid as a direct assessment program,
however, Brandman’s program administrators
established a way to define SAP as a certain
number of modules, or competencies, that a student
is expected to complete during a given subscription
period. The student must complete 67%
of the competencies in the bundle opened at the
beginning of the term in order to receive the next
financial aid payment for the following six-month term. A student that does not complete a bundle
in a previous term must catch up and finish the
remaining 33% of the bundle in the next term
in addition to 67% of the subsequent bundle. In
this way, financial aid is based on learning and
successful progression rather than time spent in
the classroom, and also keeps the student on the
path to a degree within four years.
A second regulatory requirement for financial
aid is that faculty must have substantive
interaction with students at least once a week.
Brandman’s program administrators addressed
this by requiring that the tutorial faculty, in conjunction
with the assessment graders, provide
what the DOE calls a “regular and substantive
interaction with students.” This faculty initiated
interaction with students is meaningful and varied,
conducted through required discussion
boards and assessment feedback, and occurs at
least once per week.
Besides those two specific regulatory challenges,
the program faced operational challenges
related to financial aid as well. In particular,
Brandman’s student information system (SIS)
was not designed to work with flexible six-month
terms nor different measures for satisfactory academic
progress. The integration of financial aid
with the SIS and the business office was a time consuming
process for Brandman, but the system
is expected to be ready for a spring 2015 launch,
with all components automated.
Brandman’s regional accreditor, WASC,
requires schools to have a system in place that
allows the comparison of their courses to courses
at other institutions so students have the option
to transfer. This presents a special issue for CBE
programs since the unit of currency is not a course
but rather a competency. To address this concern,
the university developed a dual transcript
model in which students receive a transcript that
shows the demonstrated competencies as well as
equivalent credit-hour courses. This comparison
is done with the use of a detailed crosswalk that
articulates the relationship of competencies to
existing courses at Brandman. The student can then use the resulting credit-hour transcript to
transfer to another college or university as well as
to the credit-based programs within Brandman.
However, since credit hours and competencies do
not perfectly align, Brandman advises students
that there is the possibility that not all courses or
competencies will successfully transfer.
Similarly, students may also transfer previously
earned college credit into the BBA program.
Students wishing to enroll in the BBA program
with some college-level coursework already
completed can work with an advisor to establish
which competencies have been met through
prior coursework. Students must have earned a
minimum grade of B– or higher to have courses
from a regionally accredited institution evaluated
for equivalency to a competency. This crosswalk
of credit-hour courses to competencies is completed
during the admission process.
Brandman University recognized that its newest
program is in relatively uncharted territory
and will likely go through some minor changes
in its initial phases. For this reason, the university
opted for a slow and deliberate launch. A total of 44 students began the BBA in October
2014 through a special partnership with over 25
employers providing 100% tuition reimbursement.
Brandman plans for the next launch of the
program to be a larger group of students coming
from organizations and companies as well as
community college transfer students.
Prior to taking the program to scale, Brandman
plans to continuously monitor and improve the
content and structure. With assistance from Flat
World, Brandman administrators are able to receive
real-time feedback from students as well as data
on their progression. If students tend to bottleneck
in certain areas, the faculty will be able to make
changes to facilitate a smoother experience. The
formative assessments provide data on what specific
areas might be left unclear to the learners,
while the summative assessments can show which
modules convey the content well and which could
benefit from improvement. The end result of this
ongoing real-time data analysis will be a CBE program
that conforms to the needs of its students
and employs the techniques and content that are
most suited to achieving positive outcomes.
It is expected that the program taken to scale
will potentially have an enrollment of thousands
of students.
The views expressed in this publication are those of the authors and do not necessarily represent those of
Lumina Foundation, its officers, or employees.
The authors of this case study were CAEL’s Richard Olson and Rebecca Klein-Collins. The case
study benefitted from several interviews and correspondence with Laurie Dodge of Brandman
University and further input from Hadassah Yang of Brandman University
Association of American Colleges and Universities. (2008). Liberal Education and America’s Promise essential
learning outcomes. Retrieved from http://www.aacu.org/leap/documents/EssentialOutcomes_Chart.pdf
Brandman University. (2015). Competency-based education: Frequently asked questions. Retrieved from
https://www.brandman.edu/cbeducation/faq
Direct Assessment Programs, 34 C.F.R. § 668.10 (2011). Retrieved from http://www.gpo.gov/fdsys/pkg/CFR-
2011-title34-vol3/pdf/CFR-2011-title34-vol3-sec668-10.pdf
Dodge, L. (2012). Brandman University adopts the degree qualifications profile. Retrieved from https://www.
wascsenior.org/files/Brandman%20University%20Adopts%20the%20Degree%20Qualification%20
Profile_January%2031%202012_final.pdf
Klein-Collins, Becky (2012). Competency-Based Degree Programs in the U.S. CAEL. Retrieved from http://
www.cael.org/pdfs/2012_competencybasedprograms
Krathwohl, D. R. (2002). A revision of Bloom’s Taxonomy: An overview. Theory into Practice, 41(4), 212–218.
Retrieved from http://rt3region7.ncdpi.wikispaces.net/file/view/8+Perspectives+on+RBT.pdf
Lumina Foundation. (2014, October). The degree qualifications profile. Retrieved from http://www.luminafoundation.org/files/resources/dqp.pdf
U.S. Department of Education. Information for Financial Aid Professionals. (2013, March 19). Applying for title
IV eligibility for direct assessment (competency-based) programs. (Dear Colleague Letter GEN-13-10).
Retrieved from http://ifap.ed.gov/dpcletters/GEN1310.html
U.S. Department of Education. Information for Financial Aid Professionals. Quality Assurance Program. (2014).
Satisfactory academic progress. Retrieved from http://ifap.ed.gov/qahome/qaassessments/sap.html
Leverage our complete white label degree program and course catalog to get online fast.Strut’s customizable course catalog provides a rapid (30 days) pathway to online courses with zero capital expenditures.Schedule a call now.
Leverage our complete white label degree program and course catalog to get online fast.Strut’s customizable course catalog provides a rapid (30 days) pathway to online courses with zero capital expenditures.Schedule a call now.
Leverage our complete white label degree program and course catalog to get online fast.Strut’s customizable course catalog provides a rapid (30 days) pathway to online courses with zero capital expenditures.Schedule a call now.