Moving away from pencil-and-paper tests

By Heather King - July 2014

Existing (and essentially school-based) approaches of assessment involve recording the extent to which learners gain particular knowledge or skills. However, in informal settings, outcomes depend on the participant’s own agenda. Measuring attainment for a pre-determined set of objectives therefore clearly has its limitations.

Michalchik and Gallagher suggest a shift in approach to assessing the outcome of informal learning activities. Their framework detects a range of outcomes, including unanticipated ones. They contrast their proposal with two existing types of assessment:

• Type 1 assessments consist of specialised activities designed to elicit specific information about a learner’s knowledge. Pencil-and-paper tests are classic examples.
• Type 2 assessments involve an outsider analysing an activity, for example by observing a class or workshop. This type of assessment does not involve any individual tests for the participants, but it is nonetheless framed within a set of pre-determined judgements about “what counts” as learning.
• Type 3 assessments—the approach the authors advocate—involves observing an activity without any judgements about “what counts.” Instead, the observation records the variety of outcomes that emerge from the interaction itself and notes any factors affecting these outcomes. Programme designers can use the results of such observations to amend and fine-tune the activity.

To illustrate, the authors refer to a model robot-making workshop. A Type 1 assessment would examine participants’ knowledge of robotics or attitudes towards engineering. A Type 2 assessment, shaped by the observers’ perceptions of “what counts,” might capture participants’ comments about their new interest in robotics. A Type 3 assessment, by contrast, would draw attention to the range of behaviours exhibited in the workshop. It might note, for example, that some students appeared more interested in painting their robots than in making them move. Although this observation may at first be disappointing to the workshop designers, it raises interesting questions: Why do participants care more about aesthetics than mechanics? How can we capitalise on students’ interests? How should we modify the workshop in future?

Implications for Practice 

Although a Type 3 approach may not tell the designers whether their activity achieved pre-determined goals, it does tell them what participants gained and what they considered to be important or interesting. Essentially, this approach involves a shift from looking at what is in the learner’s head to looking at a learner’s performance and participation.

Michalchik and Gallagher argue that the Type 3 approach is particularly suitable for assessing learning and participation in informal settings, where learners come into activities with varying prior experiences and pursue the activities in order to achieve personal goals. In a Type 3 framework, the purpose of assessment is to determine all effects the participants experience and then use these results to improve the programme. The Type 3 approach not only provides insight into individual participants’ achievements and needs, but also offers an overview of the success of the programme itself.