My aim was to support instructors to develop assessment tasks that encourage students to take more responsibility for their own learning, learn from their mistakes, explore their learning through peer conversations and to develop their professional attributes including judgement, reflection and critical evaluation.
My tool of choice when using self and peer assessment is SPARKPLUS. SPARKPLUS enables students to confidentially rate their own and their peers' contributions to a team task or individual submissions.
My recommendation is not because I am the SPARKPLUS project leader, but because I genuinely believe it is the best self and peer assessment tool available. In particular, to my knowledge it is the only tool that encourages academic honesty through its capacity to both detect suspected free riders, saboteurs and over raters and mitigate their impact on the self and peer assessment process.
Dr Keith Willey
The University of Technology, Sydney
This document has been produced to assist instructors in using SPARKPLUS in their classes. It focuses on providing supporting material including information to be included in assessment and subject guides.
These videos were produced to assist instructors and students to interpret the factors produced by SPARKPLUS.
We recommend that unless you are already familiar with SPARKPLUS that you view these videos in order. In addition, you may find it helpful to read the excerpts (reproduced below) from the SPARKPLUS user guide explaining the SPARKPLUS SPA (Performance) and SAPA (Feedback) factors.
(the following information has been taken from the SPARKPLUS user guide)
In the norm based assessment mode used to assess an individual's contribution to a team project or task SPARKPLUS automatically produces two factors.
Individual mark = team mark * Individual's SPA
For example, if a team's project mark was 80 out of 100 and a team member receives a SPA factor of 0.9 , they would receive an individual mark of 72 to reflect a lower than average team contribution as perceived by a combination of themselves and their peers. Alternatively, if not used to moderate summative assessment the SPA factor can be used formatively to assist student development.
In applying the SPA (Performance) factor we recommend that the maximum mark be capped at 100% to reflecting the maximum available mark for demonstrating the particular learning outcome or outcomes achievement. For example, if the project mark for a high-performing team was 95% and the highest contributor to this team received an SPA factor of 1.1, then without capping this student would receive a mark greater than 100% of the marks allocated for demonstrated achievement of the associated learning outcomes.
95% * 1.1 =
The second factor calculated is the SAPA (Feedback) or Self Assessment to Peer Assessment factor. It is the ratio of a student's own rating of themselves compared to the average rating of their contribution by their peers. This has strong feedback value for future development both for self-critical reflection and peer evaluation.
It provides students with feedback about how the rest of the team perceives their contribution unsullied by their own opinion. For example, a SAPA factor greater than 1 means that a student has rated their own team performance higher than they were rated by their team peers. Conversely, a SAPA factor less than 1 means that a student has rated their own performance lower than they were rated by their peers.
This shows the relationship between the three methods of calculating the SPA factor. Note the Knee plot has been slightly offset to increase readability.
SPARKPLUS allows the use of different formulas to accommodate the design of assessment tasks with different objectives. For example the above figure shows the relationship between the three formulas known as Original, Knee and Linear to calculate the SPA factor.
By way of explanation we will consider the following example. The following table shows the SPA factors for a team of four students where initially Student A only contributes half as much of the work contributed by his three team peers. In the second instance Student A contributes twice as much work as his team peers. The coordinators reported that the knee formula helped promote teamwork and fair division of the assessment task between team members. The knee formula does not reward students who might be tempted to do most of the work (Table shows a student who did twice as much work as their peers would only get an SPA factor of 1.26) while providing incentive for those who are tempted to underperform (Table shows a student who did half as much work as their peers would get an SPA factor of 0.57).
SPA factors using different formulae for groups with over and under performing team members.
|Assessor||Student A||Student B||Student C||Student D||SPA Original||SPA Knee||SPA Linear|
|Student A Contribution Half that of other Team Peer's|
|Student B, C & D||2||2||2||2||1.07||1.07||1.14|
|Student A Contribution Twice that of other Team Peer's|
|Student B, C & D||1||1||1||1||0.89||0.80||0.80|
Students have also reported that they liked the knee formula as it provided a fairer distribution of marks and sent a stronger feedback message to underperforming students than with the factors calculated using the original formula. Some students had previously expressed concern that, using the original formula, underperforming students received an inflated mark that they were satisfied with, and hence were not motivated to improve their performance for the remaining parts of a project (Willey & Freeman, 2006). For example the Table shows using the original formula a student who only did half as much work as their team peers would get an SPA factor of 0.76 and hence receive 76% of the group mark. Using the knee formula this student’s mark would be reduced to 57% of the group mark, a figure that more closely reflects their true contribution.
Willey, K. & Gardner A (2010) "Investigating the capacity of self and peer assessment activities to engage students and promote learning." European Journal of Engineering Education 35(4): 429 - 443.
Willey K & Gardner A (2009) Developing team skills with self- and peer assessment: Are benefits inversely related to team function? Campus-Wide Information Systems Vol 26 (5) pp. 365 - 378 Emerald Group Publishing.
Willey K & Gardner A (2009) Improving self- and peer assessment processes with technology. Campus-Wide Information Systems Vol 26 (5) pp. 379 - 399 Emerald Group Publishing.
Willey K and Gardner A. (2009), Changing Student's Perceptions of Self and Peer Assessment. Proceedings of the Research in Engineering Education Symposium 2009, Palm Cove, QLD, Australia
Willey K and Gardner A. (2009), Self and Peer Assessment: A Necessary Ingredient in Developing and Tracking Students' Graduate Attributes, Proceedings of the Research in Engineering Education Symposium 2009, Palm Cove, QLD
Willey K and Gardner A. (2008) Using self and peer assessment for professional and team skill development: do well functioning teams experience the benefits? Proceedings of the ATN Assessment Conference: Engaging Students with Assessment 20-21 November, Adelaide, Australia ISBN 20978-0-646-504421
Willey, K and Gardner, A. (2008) Using Self Assessment to Integrate Graduate Attribute Development with Discipline Content Delivery. Proceedings of the 36th Annual Conference of the European Association of Engineering Education (SEFI) 2-5 July, Aalborg, Denmark.
Willey, K. and Freeman M. (2006a), Completing the learning cycle: The role of formative feedback when using self and peer assessment to improve teamwork and engagement. Proceedings of the 17th Annual Conference of the Australasian Association for Engineering Education, 10 -13th December 2006, Auckland, New Zealand.
Willey K, and Freeman M. (2006b), Improving teamwork and engagement: the case for self and peer assessment, Australasian Journal of Engineering Education. Online publication 2006-02