An Empirical Study on Alternative Assessment of Portfolios Based on Saaty’s Analytic Hierarchy Process (AHP)

 

Sheila Cheng and Heng Loke Siow

Asia e University
Malaysia


Digital knowledge has been recognized as an indispensable asset in today’s world. As educators, how can we help students to acquire 21st century skills? Digital learning technology plays an important role in assessment. Various assessment formats have been recommended for engaging and motivating students’ learning. A portfolio/e-portfolio has been recommended in many studies (Kampylis, Punie, & Devine, 2015; McDonald, 2011; UNSW, 2019) as one of the effective ways to document and communicate students’ learning process. However, the portfolio is a very subjective form of assessment. This paper attempts to conduct an empirical study on alternative assessment of portfolios based on Saaty’s Analytic Hierarchy Process (AHP). The assessment framework, called “3AHP”, was established against the course learning outcomes (CLOs).

 

Two sets of portfolios submitted by undergraduate students for two different courses which were applying for course exemption through Accreditation of Prior Experiential Learning (APEL) were assessed respectively by both the traditional way of rating scales with rubrics and 3AHP. A comparison of rating scale assessment versus 3AHP is discussed. Assessment criteria for both rating scales and 3AHP are also considered.

 

The results of this study indicated that both rating scales and 3AHP generate similar assessment results.

 

The 3AHP assessment approach simplifies the portfolio assessment processes. In addition, the 3AHP approach standardizes and minimizes the variation in portfolio assessment among different assessors. This approach is generic in nature. It can be applied to assess the portfolio of formal, informal and non-formal learning; and it can also be implemented in different disciplines and simplify the task of assessors at institutions of higher learning.