Gray, Janice --- "Review: Improving Criteria And Feedback In Student Assessment In Law" [2001] UTSLawRw 16; (2001) 3 University of Technology Sydney Law Review 232
Richard Johnstone, Jenny Patterson And Kim Rubinstein, Improving Criteria And Feedback In Student Assessment In Law, Cavendish, Sydney, 1998
Reviewed by Janice Gray
LEGAL ACADEMICS are not always easily encouraged to pick up a book on the subject of teaching and learning. Some are so engrossed in the substantive law and policy which lie at the heart of their teaching and research that they tend to devote their time to maintaining expertise in those areas. Indeed the interviews with two outstanding teachers which appear elsewhere in this volume demonstrate that even superior teachers do not always see it as a priority to focus on educational theory. Instead there is a commonly held view that through trial and error each teacher is able to sift through, calibrate and decide on the best of alternative approaches to various aspects of teaching such as goal setting in the form of determining aims and objectives, delivery modes, evaluation, assessment and feedback. It is probably true that a personalised, pragmatic approach such as this can yield acceptable and sometimes excellent results. However, it is the view of this reviewer and seemingly the authors of Improving Criteria and Feedback in Student Assessment in Law that the task of deciding which is the most appropriate method to use for any given aspect of teaching (whether developing objectives or assessment schemes, for example) is assisted greatly by a more formal exploration of why we wish to embark on the task in the first place and how others have approached it previously.
That is where this book, Improving Criteria and Feedback, is particularly useful. One of its great virtues is that it provides a bridge between theory and practice. The first half offers an analytical and conceptual account of the key issues surrounding assessment and feedback while the second offers practical examples of lecturers’ attempts to address these issues.[1]
As explained in the preface, the book is one of the outcomes of a project funded by a Committee for the Advancement of University Teaching (CAUT) grant. An application for the grant was submitted because a number of academics in the Law Faculty of Melbourne University were concerned that procedures were not in place for students to receive feedback on all their assessment tasks. This book explores why there should be appropriate assessment criteria, what assessment could consist of and why feedback about assessment tasks is necessary. Finally, it specifically considers what this feedback should incorporate.
One of its endearing features is that it is not a ‘know it all’ book. This book is not written in a didactic style and it does not suggest that it has all the answers. Instead it raises issues, talks about problems associated with dealing with the issues and then offers the best practical solutions to the problems that the authors know about. It does not purport to say that these are perfect or fail safe. Such honesty is refreshing.
The preface of the book attempts to engage the reader in a dialogue about (a) the need for effective assessment and (b) communication of feedback regarding that assessment. The authors openly face the criticism that is so regularly and often very justifiably levelled against such projects, i.e. that any requirement of teachers to clarify criteria for assessment and to provide detailed feedback is too time consuming. This thread runs all the way through the book and where suggestions are made about implementing feedback schemes such as individual feedback interviews with students, recognition is given to the fact that in large classes, usually where the lecture/tutorial format prevails, this option would be too demanding for the teacher involved.
Interestingly, the authors offer a bait in their preface for the reader to stay connected to the discussion by suggesting that implementing the approach to assessment criteria and feedback explained in the book would permit teachers to generate material that could be appended to their teaching portfolios and used when applying for selection, confirmation or promotion. For some that may be compelling enough, in itself, to read on.
In short the basic proposition which underpins the book seems to be that students and staff alike would benefit from the use of a wide range of assessment modes with feedback about performance being actively communicated to students in a variety of ways. Concern about assessment and feedback came to the fore after the authors observed students’ responses to the issues over some years. In brief, students acknowledged that whilst many individual teachers offered clear aims and objectives, generally speaking there had been insufficient feedback on essay criteria and very little structured feedback on end of session examinations. The authors’ view is that the effectiveness of assessment is diminished if there is not satisfactory feedback to students on their performance in those assessable tasks.
With that brief in mind the authors set about exploring the issues in chapters titled ‘Educational Principles for Designing Assessment’, ‘Formulating Objectives’, ‘Assessment Task and Criteria’, ‘Provision of Feedback’ and ‘Evaluating Feedback in the Context of the Overall Evaluation of Teaching’. The chapters demonstrate a clear sequential development, with the earlier ones laying the basis for material in the later ones. At this point it is worth selecting some of the key issues which are raised in these chapters and discussing them briefly.
Chapter one offers a definitional distinction between formative and summative assessment, singling out the credentialling aspect of summative assessment as of key importance.[2] This point is explored in order to demonstrate how summative assessment is not intended to assist students in improving their performance. However, the authors point out how in many circumstances the line between formative and summative assessment is blurred, with summative assessment offering formative and diagnostic aspects. The purpose of assessment is then reviewed. Its purpose is elaborated as being a motivator of student learning, defining the curriculum, aiding the structuring and consolidating of student learning, and providing feedback to students.[3] Through discussion of Laurillard’s template, the authors highlight the fact that feedback can be given at any stage of the learning process and need not necessarily be formal.[4] Statements such as the latter, although fairly obvious, have the effect of validating those moments when teachers may take five or ten minutes at the beginning or end of a class both to give feedback and receive it from students. The notion of exchange and dialogue is encouraged by an awareness that feedback need not necessarily be formal.
Interestingly, Johnstone et al. raise the issue of the responsibility of students to digest and use the feedback that is given to them. This is a point that is often overlooked in discussion of feedback and the authors do well to draw attention to it. If feedback is to be seen as a genuine exchange it must necessarily involve a willingness on the part of students to respond to comments and to take them on board. In other words students must be willing to process the feedback if it is to serve any worthwhile purpose. Those teachers who have ever worked in institutions where exams scripts were available to be collected and taken away by students would be able to identify with the frustration felt when it is realised that the scripts they have so laboriously commented on in writing remain uncollected. Perhaps Johnstone et al. might also have mentioned a related but different point and that is how if feedback is to involve a genuine dialogue involving explanation and ultimately student improvement in assessment tasks, students need to be sincere about that dialogue. This means that feedback can really only take place when students are not angry or shocked by their results. If the feedback is of the oral interview type, it is perhaps best left until the student has calmed down and had a chance to re-read the question and answer.
Among other things chapter three of the book delineates the fundamental link between objectives and assessment. Effective assessment is the means by which one determines if the course objectives have been satisfied. This connection is of key importance but it is unfortunate that university administrators sometimes lose track of this significant connection when setting deadlines. For example, where one has to provide details of a course’s assessment schedule in, say, the October preceding the March commencement of a course, the link between the objectives and assessment can become a little tenuous. Deadlines such as this preclude teachers from taking into account the previous session’s student evaluations when developing an assessment schedule for the following session. Any formal feedback teachers may receive from students, through evaluation questionnaires on the effectiveness of the assessment schemes, arrives after the course outline has been prepared. Hence teachers cannot tailor their assessment schemes according to whether previous students found them successful means of determining the achievement of course objectives. The disharmony between best teaching practice and administrative efficiency is not a new issue. It necessarily underscores many a discussion on teaching theory and practice.
Another useful point that this chapter raises is the diversity of objectives which can usefully form part of the law curriculum, e.g. intellectual objectives and practical or skills objectives. In particular, it is notable that Johnstone et al. include discussion of affective objectives as well. The authors list in this category, among others, the development of self confidence, independence and initiative as well as an awareness of the impact of law on people’s lives. In an era in which economic rationalism has been a hallmark, it is a wonderful privilege to be able to deal with the development of the affective domain. It is to the authors’ credit that they have not lost sight of such objectives, which often underpin the social justice element of the study of law.
Yet another very worthwhile point that this book makes is that objectives need to dovetail together so that no one course seeks to cover all available objectives. For example, it is suggested that in the first year of a course objectives might concentrate on knowledge and understanding of a substantive area of law, critical perspectives, basic case reading skills, the application of case law to new fact situations, statutory interpretation and legal research. In second year the objectives might include the development of case reading skills, legal analysis and statutory interpretation as well as formal legal writing and oral presentation skills.[5] In this way there is a gradation of objectives with one building on another. Clearly assessment schemes need to reflect the different level of skills that each objective requires. It is also noted in this chapter that the different assessment schemes associated with the testing of these objectives will require different types of feedback. For example a true/false or multiple choice test will not necessitate detailed assessment criteria because the grading will be fairly mechanical.[6] By comparison a research essay will require quite detailed assessment criteria because there is a very great range of attributes that the teacher could be seeking to assess.
The chapter on the provision of feedback is both informative and helpful. It suggests that if students do not receive satisfactory feedback they may not know why they did well or poorly in any given assessment task. They might not also know where they are lacking and how to improve. Consequently they may become demoralised.[7] Indeed the chart entitled ‘Principles to guide feedback on assessment tasks’ is most useful in directing the reader to the nitty gritty aspects of feedback that teachers must keep at the forefront of their minds.[8] It is the sort of chart that teachers would do well to revisit regularly.
It is evident that one of the recurring themes in the book is that best feedback practice revolves around timely feedback that is personalised. Whilst the authors offer examples of feedback sheets which require the lecturer to tick the box, they constantly reinforce that the best feedback for written work is actually writing comments directly on the students’ work. They also acknowledge the efficacy of personal interviews with students in order to give exam feedback. Further, they highlight the failings of the trial mass examination feedback session held in a large lecture room where students were given their scripts, exemplar answers and a modified marking guide which explained how the criteria for assessment should have been met. In short, a considerable number of students complained that this approach provided insufficient feedback. That personalised feedback is the best necessarily raises the issue of how time consuming this task is and how under-resourced universities presently are. Although the authors are clearly aware of the constraints on teachers’ time they continue to suggest the use of a wide range of assessment tasks and a wide range of feedback relating to those tasks. Yet at the end of the day the inescapable truth must be faced. Teachers are involved in a balancing game. Does one offer students fewer assessment tasks with detailed and personalised feedback or more assessment tasks with a range of feedback methods designed to impose fewer time constraints on the teacher?
In an ideal world things would be different, but particularly for those law teachers who teach 12 or more hours each week, co-ordinate subjects or courses, fulfil administrative duties, attend meetings, maintain an active research and publication profile and fulfil expectations as to community involvement, the message that teachers should employ a diverse range of assessment tasks on which a wide range of personalised feedback is given may well fall on deaf ears.
Perhaps the best way to read Improving Criteria and Feedback is to see it as offering suggestions for what one might do if universities were appropriately funded and resources were available which permitted academics to fulfil more readily the obligations associated with their positions. Until such time as universities are better resourced most academics will be forced to dilute a number of the suggestions in this excellent book, modifying them to fit an era of the academy’s impecuniosity.
[1]Unfortunately space does not permit a detailed analysis of the practical approaches contained in the second part of the book but they are recommended as being very worthy of serious consideration. They include examples of marking criteria for exams, self reflection checklist and feedback sheets, a marking and criteria guide for essay writing, examination information, explanatory notes on how to write a case note as well as a research assignment feedback sheet.
[2]Formative assessment generally offers a diagnostic approach and seeks to assist students improve their skills and competence. Summative assessment is more likely to offer a final account of how well students have demonstrated skills and knowledge. It usually involves a public statement of the student’s perfomance. See D.R. Sadler, ‘Formative Assessment and Design of Instruction Systems’ (1989) 18 Instructional Science p.119.
[3] Johnstone, R., et al, Improving Criteria and Feedback in Student Assessment in Law, Cavendish, Sydney, 1998 pp. 7—12.
[4]Laurillard, D., Rethinking University Teaching: A Framework for the Effective Use of Educational Technology, Routledge, London, 1993, p.85.
[5]See Johnstone, R., et al, Improving Criteria and Feedback in Student Assessment in Law, Cavendish, Sydney, 1998. pp. 22—23.
[6]Ibid p.32.
[7]Ibid p.39.
[8]Ibid p.40.
[1]p.14.
[2]p.16.