Keyes, Mary E; Whincop, Michael J --- "The Moot Reconceived: Some Theory and Evidence on Legal Skills" [1997] LegEdRev 1; (1997) 8(1) Legal Education Review 1

THE MOOT RECONCEIVED: SOME THEORY AND EVIDENCE ON LEGAL SKILLS

MARY E KEYES* & MICHAEL J WHINCOP**

I.  INTRODUCTION

One of the major contemporary challenges to legal educators is the effective incorporation of skills training into the undergraduate law curriculum. Many of the most interesting issues in curriculum development in law schools are concerned with the identification of skills that should be taught, the subjects with which they should be associated, and the particular means of teaching by which skills can be conveyed to the undergraduate.

It is possible simultaneously to affirm the importance of skills training, while feeling uneasy about legal educators’ capacity to give reasoned answers to the above curricular questions. These doubts arise from a lack of useful applications of assessment theory to educational practice, and most particularly, legal educational practice. The way in which teachers assess the material which they teach has far-reaching implications for student learning. Assessment not only influences what, and how, a student learns; it also has pervasive “feedback effects for how one teaches. We argue that this point is especially germane to skills training. Assessment theory informs the structure, content and processes of skills training.

In order to demonstrate these claims, we study the relevance of assessment theory to one long established method of skills training — the moot. Our argument is that moots represent a useful means by which to train students in skills. Those skills are by no means limited to advocacy, the most obviously required skill. However, the traditional form of the moot, as a means of assessment, can be an inefficient means by which to develop student skills. The reasons for this inefficiency are most clearly appreciated by exploring moots through the lens of learning and assessment theory.

Part II describes the major contributions to the educational literature on assessment. The conclusions of Part II are applied in Part III to critique the concept of the traditional moot, and to develop an alternative model of mooting. We describe this as a formative moot, because it uses formative assessment techniques to influence the quality of student learning. Part IV reports on the results of an experiment involving the use of a formative moot in a compulsory subject at the authors’ law school. Evidence is qualitative and quantitative. It endorses the claims we make concerning the limitations of the traditional moot concept, and the advantages of changing the parameters of a learning activity in order to improve learning outcomes, in particular by embedding formative assessment in the exercise. Part V is a conclusion. It comprises our analysis of the implications of our work for legal education and skills training.

II. A THEORY OF ASSESSMENT

It is apparent that assessment should be soundly based in educational theory. However, current theories of student learning do not always clearly articulate their implications for assessment. Conversely, the literature on assessment is often vague as to its precise theoretical foundations. For these reasons, assessment has languished as an “afterthought”1 — an irony, given that assessment is frequently acknowledged to reveal the “hidden curriculum”.2 The purposes of this part are as follows: first, we consider various theoretical insights into the nature of student learning. Second, we discuss the connection between assessment and learning. We describe a tension between lifelong learning and the discrete subject paradigm which prevails in law schools, and the power of progressive and formative assessment to mediate that tension. Third, we consider the principles behind the recent advocacy of. legal skills. We conclude that the key to integrating skills into the curriculum should be based in the considered planning and selection of learning objectives, and deriving assessment strategies from these objectives.

A. Cognitive Theories of Learning

Most modern learning theory assumes the effectiveness of interventions by teachers in the institutional learning process. The practical implications of learning theory are therefore to indicate appropriate interventions by the teacher which are intended to facilitate high quality learning. Cognitive theories, based in psychology, have had a pervasive influence on the education literature. These theories focus on the process of learning from the learner’s perspective, so advocating “student centred” approaches. They provide the foundation for constructivism, which describes learning as an active process of “constructing meaning and transforming understandings”.3

Situated cognition theory emphasises the significance of context to learning.4 It has two implications: first, teachers should consider the influence of context on learning; and, second, teachers should manipulate context in order to facilitate student learning. Cognitive theory emphasises the importance of different learning, or cognitive, styles, an inevitable consequence of focussing on the individual learner. The theory provides means for contextual manipulation in order to maximise meaningful learning.5

Experiential learning theory describes how students actively engage and participate in their own learning through experience. The active participation in concrete experience is regarded as being highly significant to the learning process. Experiential learning theory also draws on the observations that students reflect upon experience and that reflection is used to construct meaning6 The reflective component of learning is thought to be central to the development of certain abilities, especially generic “professional” characteristics.7 The practical implications of experientialism are that teachers ought to allow students to engage in concrete experience and to encourage reflection by students.8 Experiential learning theory has been influential in legal education in the USA since the 1980s, especially in debate concerning skills development.9

Cognitive learning theories emphasise the ongoing nature of learning, and conceive institutional education as being one part of that ongoing process. One of its main roles is to develop abilities to continue learning outside the institution. This demands that the abilities developed have wide application and transferability, so that students are able to adjust to new developments, contexts and technologies.10 The focus here is not disciplinary.11 Related to these concerns is the recent focus in tertiary education on the minimisation of the “gaps” between institutional and “real world” learning, by observing and describing real world activity, and replicating it in the institutional environment.12

Cognitive apprenticeship theory synthesises theories of situated cognition and experiential learning, with the need to close the gap between institutions and the real world. It focuses on the existence in the learning environment of an “authentic” professional culture.13 It posits the efficacy of a traditional model of apprenticeship as the best method of learning,14 and assumes that apprenticeship-like training can be replicated in formal education. The theory asserts that learning will be most effective where students undertake “authentic” tasks. Authentic tasks in institutionalised legal education are those resembling tasks undertaken by lawyers in private practice.15 Cognitive apprenticeship thus endorses collaborative activities, as well as tasks such as skills based activities. Skills training is studied below, but first the connections between learning and assessment are examined.

B. Assessment

The development of theoretical work on assessment has been overlooked in the current focus on learning.16 This tendency applies to legal education as well as to other disciplines. Most writers agree, consistently with the focus on student learning, that assessment is an integral aspect of the learning process.17 There is a tension in assessment practice between the perspective of the learner and the perspective of the teacher. The pragmatic task of reconciling constraints applying to the teacher with the implications of learning theory from students’ perspectives gives rise to an argument in favour of progressive assessment, which we develop below.

1. The Process of Learning: The Student’s Perspective

Cognitive learning theory emphasises that learning is an ongoing experience. Experiential theory emphasises the importance of reflective aspects of learning. These two principles have important implications for assessment. The ongoing nature of learning can be considered across a variety of time periods. However, the relevant time periods from the perspectives of students’ experience of learning, and teachers’ effective control, are quite different.

The literature on student learning indicates that from students’ perspectives, learning is not neatly divided into self contained, semester or year long units which have no further impact on their development, learning and understanding. However, the teacher’s direct influence is very much limited to facilitating students’ achievement of learning objectives in individual subjects.18 Given these premises, the most effective way for the individual teacher to influence learning is by using progressive assessment strategies that provide feedback.19 This conforms to the ongoing nature of learning.

2. The Teachers’ Perspective: The Importance of Subject Design

Most of the literature on assessment focuses upon practical suggestions for subject design. In keeping with the convention we take the individual subject as the proper unit of analysis.20 The teacher responsible for subject design exerts a fair degree of influence over the particular assessment schedule, subject to varying external influences. At the teacher’s level, relevant influences include the teacher’s experience, their philosophy of learning and teaching,21 their familiarity with educational theory and design, increasing time pressures, and their perceptions of external expectations. Significant external influences include faculty, school and institutional policies on assessment, explicit requirements and informal expectations of potential employers, professional bodies, and other “stakeholders”.

The focal point of subject design is the articulation of learning objectives, which should found every aspect of the subject.22 Rowntree describes these as: “the skills, abilities, knowledge and understanding in which the teacher intends that students should improve as a result of his interventions.23 Subject design should ensure consistency between learning objectives, the modes of subject delivery, and assessment.24 Assessment should enable students to achieve learning objectives.25 It is commonly assumed that communicating to students the rationale underlying subject design, the publication of learning objectives, and explanation of assessment criteria are effective in assisting students’ achievement of the learning objectives. 26 Many writers have commented on the effect of formal assessment on students’ instrumental and strategic approaches to learning. Making an activity assessable increases its value to most students, and hence their motivation and application?27

However, we counsel against relying on these general prescriptions in subject design in isolation. Without an understanding of the implications of learning theory for assessment, and the theoretical foundations of assessment, subject design can become obsessed with proceduralism. We explore below the implications of learning theory for assessment.

3. Ongoing Learning Activities: Formative and Summative Assessment

The distinction between formative and summative assessment relates to the purpose of assessment exercises. Assessment is formative where it occurs as part of a progressive learning exercise, and where the main purpose is to facilitate student learning.28 Formative assessment usually incorporates the provision of feedback as activities are completed.29 Some aspects of earlier activities are developed in later assessable activities. Theories which emphasise the ongoing nature of learning clearly support, if they do not mandate, the use of formative assessment.

Summative assessment, in contrast, refers to assessment which is sometimes described (quite innocently) as terminal.30 The abilities and competencies examined in summative assessment are not subsequently assessed within the subject. Summative assessment reports on and certifies the “achievement status of a student”.31 The intended recipients of reports about students’ achievement status are generally external audiences.32 Summative assessment is often assumed to occur at the end of a subject, but this is not necessarily the case. The issue is whether the specific content and processes are subsequently assessed. Therefore, the use of more than one type of assessment instrument in an assessment schedule does not necessarily mean that the assessment schedule is progressive or formative.

The use of summative assessment to improve student learning in a directed way is difficult.33 The use of summative assessment alone assumes that assessment tasks can be designed in a way that will positively assure that students will achieve the learning objectives without input from the teacher.34 Summative assessment commits the teacher in advance to a particular course and makes it difficult to respond to, direct or assist the learning which is actually occurring in the performance of the assessment activity.35

4. Feedback

Feedback is evaluative information which may be used by students in reflecting upon and improving their performance in a learning activity. The word “feedback” suggests that the information is externally provided.36 In fact, there is a growing body of opinion that students should be expressly instructed in techniques and strategies for gathering evaluative information (including self evaluation) and using it in self reflective processes.37 A discussion of peer and self evaluation is beyond the scope of this article; we intend to focus on feedback which is provided explicitly by teachers.

Although it is currently fashionable to encourage teachers to provide feedback, this inevitably entails the expenditure of considerable time and effort. In order to make the best use of the teacher’s costly resource, namely his or her time, it is essential to be clear about the theoretical foundations of feedback. As explained above, experiential learning theory suggests that students do, and should be encouraged to, reflect on experience. Feedback is a significant way in which teachers may influence student learning. Feedback encourages self reflection and provides useful material on which to reflect.38 By providing feedback, the teacher may directly assist students in achieving the learning objectives of the subject by providing responsive information about aspects of their learning.

Resolving the tension between the limits of teacher influence and the ongoing nature of learning indicate that progressive activities are the best form of assessment. Feedback by teachers will be most effective where it is provided in response to early activities in progressive assessment schemes, aspects of which are developed in later learning activities in the subject. In the absence of well planned and implemented course design, in which connections are developed between learning activities and assessment, the value of feedback to students may not be worth the cost to the teacher.

5. The “Real World” and Assessment

We noted above that cognitive theory’s prescription — that the gap be closed between the academy and the real world — has influenced education in practice. Replicating realistic activities is thought to be appropriate to the design of assessment as in other aspects of learning and teaching, although it should always be posterior to subject design issues of setting learning objectives and developing assessment by reference to those objectives. Thus formal assessment like real world practice should incorporate (and therefore assess) more than one type of activity.39 Real world work is also likely to be progressive, and the use of feedback for improvement is often observed to be part of expert practitioners’ repertoires.40 Group and co-operative activity is frequently noted to be far more representative than the individualistic and competitive nature of most assessment tasks used in higher education?41 The attempt in legal education to impart professional verisimilitude to curriculum is dominated today by skills training.

C. Skills Education

The deficiencies of exclusive reliance on summative assessment in legal education have been extensively discussed, as part of broad-ranging critiques of the content of law curricula, as well as modes of teaching and learning.42 The lack of attention given to skills development has been important to these criticisms.43 In recent years, there has been a noticeable interest in and commitment to the inclusion of legal skills education into the law curriculum in Australia, and many law schools have incorporated a dedicated skills component in their curricula.44 Our belief is that practice has far outstripped theory.

Two related conceptions of skills appear in the literature. The first concentrates on identifying with a high degree of specificity the “skills” aspects of learning and teaching activities. Learning and teaching activities may be described by providing an inventory of the content and processes addressed in individual courses and subjects. This approach also has normative content. The description of practices permits identification of material and processes which should be considered for inclusion in formal legal education.45 The types of skills identified embrace those which are legally specific, those which are legally based but arguably transferable, and those which are not legally specific.46 This conception contributes to subject and course design and evaluation. We return to its deficiencies in a moment but it is worth noting that this conception shows that every aspect of legal education incorporates “skills” components,47 and therefore that it is unhelpful to describe skills as though they were separate. In this approach, the significance of skills education to law becomes obscure, because of the wide application and transferability of many of the skills. This tendency is controlled and contained by the second approach.

The second approach is related to closing the gap between the academy and the legal profession, so that academic education is directed and designed self-consciously to prepare students to join the profession.48 Skills-based learning activities in this approach are more “practical” than the traditional methods of large class, non-interactive lecturing.49 This approach posits that learning activities should be modelled on the activity types perceived to be important in practice.50

In the late twentieth century, formalistic attitudes to law and education have been successfully challenged so that both substantive content coverage and method of delivery have been revolutionised. Coverage has been expanded from a narrow doctrinal focus to include interdisciplinary material and theoretical perspectives. Understanding of the content of legal education has been improved by articulating its underlying skills components and its analytical processes. The first conception of skills reveals the close relationship between content and process — for example, learning legal doctrine has implications for the development of skills such as research, legal reasoning, analysis, and synthesis. In relation to methods of delivery, the first conception of skills has been useful in identifying the various components of learning and teaching activities. The second conception of skills has facilitated debate about the purposes of legal education, examined some parts of the academy/profession relationship, and expanded the repertoire of teaching activities.

Despite these positive contributions to the progress of legal education, both conceptions of skills have problematic aspects. The first conception gives insufficient attention to ordering the catalogued skills in terms of importance, or to appropriate methods of teaching these. It lacks the conceptual apparatus needed to transform description into prescription. In the second conception, too many assumptions are made (or too few assumptions are rigorously justified) about the types of realistic activity which ought to be included in formal education. The key to improving the quality of discourse on skills is understanding its linkage with learning objectives and assessment. Determination of a subject’s learning objectives is logically anterior to consideration of the subject’s content and processes, including skills aspects.51 It follows that teachers should be less concerned about ensuring that every skill in the inventory is covered, and more concerned to identify learning objectives. When learning objectives are clear, they will establish parameters for specific learning activities. Since skills training is a learning activity, it must be responsive to these parameters. However, it can be seen from the discussion of assessment that the extent to which skills training can accomplish learning objectives depends in large measure on how it is assessed. Because the acquisition of skills is envisaged by teachers as a lifelong learning exercise, formative assessment is of great practical importance. This is because it gives the instructor the ability to appraise the way in which a student’s skill level is developing, and, if properly specified, it can beneficially influence that development. In the next part, we address this theme in mooting. A concurrent theme is the utility of intentional manipulation of the usual procedure and incidents of skills activities. Situated cognition and experiential theories indicate that such manipulation can improve the quality of student learning.

III. THE FORMATIVE MOOT: CONCEPT AND PROCEDURE

A. Introduction

Part II suggests that teachers need to consider three ideas when they set skills activities. First, there is the need to consider the possibility of improving learning through provision of formative feedback. Second, the parameters of the skills activity may be manipulated in order to develop selected skills more efficiently, or to respond to different cognitive learning styles. Third, the teacher must be sensitive to the relationship between the skills activity and the subject learning objectives. These three principles are used in this section to identify a number of limitations of the traditional moot format. These limitations potentially compromise the quality of student learning and the extent to which relevant skills are developed. Beyond critique, we develop an alternative procedure for mooting — the formative moot — by which these limitations may be addressed. A cautionary note must be sounded, however. Although the formative moot is by our hypothesis theoretically justified, the form which we envisage it as taking is partially determined by the learning objectives of the subject in which it is implemented. Different subjects and different learning objectives will dictate some differences in emphasis and procedure. We specifically address the formative moot’s wider application in section C.

This Part will take the following form. Section B identifies four skills which are central to the moot. Because the particular learning objectives of the subject had important influences on our thinking, we identify them in section C. The focus here is to identify how we related learning objectives to skills activities. The remaining sections in this paper deal with limitations on the traditional moot. This discussion is influenced by the three ideas we have referred to — formative assessment, contextual manipulation and the nexus with learning objectives identified in section C.

B. Skillsets

Mooting presents the opportunity to develop a number of important skills.52 First, there are skills of facility with doctrine. These include the ability to conduct legal research, to apply legal principles to a factual situation, and to formulate a submission for oral (and sometimes also written) delivery to the court. Second, there are skills of advocacy. These differ primarily from the first set of skills because they are primarily rhetorical — they concern a student’s ability to persuade. Third, there are strategic skills, perhaps the least well recognised skill component. Knowing how to run a case is frequently not restricted to legal and rhetorical abilities. Strategic considerations may suggest that a side does not run every possible argument; there may be advantages in conceding some aspects of the case. Fourth, there are skills of co-ordination within groups. We show below that the traditional moot often fails to give its fullest attention to ways in which these skills can be developed. Some of these failures derive from assessment considerations, although others do not.

C. Context and Learning Objectives

The subject in question where skills issues and learning objectives had to be confronted was a single semester, compulsory course on income taxation. That course had three hours of classes a week (including small group tutorials) a quarter of which were devoted to teaching stamp duty. The subject had 104 enrolled students. In the law school program, students generally undertake one moot each year, always within the context of a particular subject.53 The difficulty of moots increases incrementally from year to year. We therefore were able to work on the basis that students had experienced two previous moots.

An income tax subject condensed into twenty hours of lecture group classes faces the problem of navigating between the Scylla of wide but superficial coverage, and the Charybdis of narrow and detailed coverage. The choice between these options was not unconstrained. A compulsory subject must cater to those who would not choose to do the subject by supplying a wider overview, from which all students can take away some knowledge. This tendency towards width was reinforced by the thematic link with the companion subject, Property Law, which is taught over 100 hours of lecture group classes a year. The thematic linkage aims to develop a contextualised understanding of how the law regulates property and property transfers. It followed that the main obstacle to be fought in the income tax course was the problem of superficiality.

This was combated by resolving that the assessment in the subject should encourage deeper learning of an integral aspect of the subject. That demanded that the assessment would require students to examine both the formal framework of the Income Tax Assessment Act 1936 (Cth) and the tensions between transactional form and substance which pervade the case law interpreting the Act. The doctrinal areas selected to do this revolved around the central income/capital distinction. This is a significant area, since the case law up until the 1980s often tended be highly formalistic. These tendencies were attacked in two ways, by the expansion by the High Court of Australia of the income concept in the Whitfords Beach54 and Myer Emporium,55 and by the legislative enactment of a capital gains tax in Part IIIA. Assessment therefore required students to examine problems within these areas. Obviously, there was a choice for achieving those learning objectives. It could be done by an examination, or by an assignment. However, the process of the formative moot suggested a means by which students could be expected to undertake deep learning in a way which alternative means of (summative) assessment would not permit. Given that the assessment task was the means by which this deep learning was to be encouraged, it was important that it embrace formative assessment techniques to examine, and if necessary, to correct, the nature of student learning that occurred. Finally, the skills required for examinations and assignments could, by means of manipulating the activity’s parameters, be achieved by the moot. These skills are, in the case of exams, the need for accurate and orderly responses under pressure; and, in the case of assignments, the need for good written and research skills. The following sections examine specific aspects of the formative moot which enables it to do these things by means of methods which the traditional moot lacks.

D. Formative Assessment

Like the one-off assignment, and the terminal examination, the traditional moot is inherently summative. If the student gets any feedback, it is usually too late to be of any use, because it comes after the student has performed all requirements of the exercise. No attention can be paid to student learning. In some ways, the moot can almost produce distorted feedback. There is a long held perception, which may not be incorrect, that a “good” moot involves a sustained attack on an advocate’s case by the judge. Marginal students who are unsure of how much they have learnt by the time they rise to their feet, can construe a barrage of questions as a sign that the argument is wrong. A student who has erred in some aspect of preparing the case rarely has a second chance of redemption in the moot. “Error” itself is an imprecise, or at least a subjective, concept. Judges have certain priors about “correct” arguments. Judges encountering an argument for the first time in the moot court may reject an unorthodox but arguable case as simply wrong.

The most logical way to overcome these problems, and to incorporate formative feedback into the process is to have two (conceivably multiple) hearings. The initial hearing provides an opportunity for the judge/assessor to provide feedback to students on their work within the situated context of a court hearing. That feedback is for use by teams in the preparation of their submissions and advocacy for subsequent hearings. The team can therefore be assessed on the extent to which they used that feedback. This enables teams to work (and therefore learn) more efficiently Specifically it permits the following. First, submissions that one side intends to concede do not have to be the subject of extensive and unproductive work by the other. Second, the judge/assessor can direct a side to abandon arguments that he or she perceives have little likelihood of succeeding. Third, the judge has an opportunity to consider unorthodox arguments before the principal hearing. Fourth, the judge can indicate the parts of the argument that he or she regards as being crucial to the case.

The divided hearing concept also provides an opportunity, which the traditional moot lacks, for developing strategic skills. With multiple hearings, the assessor can give a less detailed set of initial facts. This is because teams have an opportunity at the first hearing to seek directions as to facts that they perceive that they need, having regard to their understanding of the law.56 This process has three advantages. First, it enables the assessor to gauge the extent to which students understand the law and its implication for particular facts. Second, this method gives students a greater degree of flexibility in how they learn. This takes account of different cognitive learning styles. It permits students to take part in the formulation of their learning tasks. The judge can prepare for the students at the later hearing, rather than the reverse approach which de facto characterises most traditional moots in the authors’ experience. Such an approach is obviously student-centred. Third, because moots are adversarial, students have an opportunity to compete in the formulation of directions. They can resist or object to a direction, or seek to have further directions made which suit them. This enables a greater degree of competition and an opportunity for strategy which generally elude the conduct of the traditional moot.

Encouraging students to think more about facts increases the validity of the moot as a context in which learning occurs. A moot in which facts are taken as immutable causes problems. First, it enhances undesirably the law’s formalistic pretensions; that is, the law is a series of principles operable on the basis of minimum facts. Second, precluding students from thinking about facts limits the potential of mooting to implement cognitive apprenticeship theory through professional enculturation. The moot becomes obsessed with a narrow legal question. Although a few appeals may resemble this situation, they are unrepresentative of litigation processes. Even in appeals, courts occasionally permit new facts to be admitted. By contrast, placing the onus on students to seek factual directions replicates the process by which lawyers consider the evidence they need for their case. Because directions are made early in the process, this encourages students to think about facts from the start. Third, students restricted to facts in the question may attempt to read them unrealistically and draw baseless inferences from them.

Similarly, students can seek directions about particular legal issues. For instance, students might ask whether a court will entertain submissions seeking to challenge the authority of a key case. In the absence of a preliminary hearing, such arguments are fraught with danger — the students’ effort in preparation might be wasted; alternatively, the cautious student who does not wish to waste effort might be punished for not having argued such a point where the judge expected this to occur.

Most importantly, the first hearing provides an opportunity for the interchange of ideas between teams. This enhances the ability of teams to prepare submissions which not only establish a case, but which respond to the other team’s arguments. One might regard this as embedding adversarial advocacy in conception. The giving of directions which enhance the responsiveness of team submissions increases the amount of formative feedback that students can use in legal analysis and advocacy.

In the implementation of the formative moot concept, described in Part V, the two hearings were differentiated as a “directions” hearing and a “main” hearing, the former held about a fortnight before the latter. The name of the directions hearing reflects and is very loosely modelled on the Federal Court trial management procedure. The directions hearing was the occasion in the implemented moot for delivering a formalised type of feedback within an advocacy-oriented context.

In order to prepare for the directions hearing, student teams were required to submit a brief outline of argument of two or three pages, indicating the legal arguments that were intended to be advanced and the authorities that were to be relied on in order to establish those arguments. On the basis of the outline for the opponent teams, the judge prepared, in advance, a written summary of draft directions. This summarised the arguments of each team into précis form, and provided a list of directions. These were primarily in the nature of key items of feedback to each team on the basis of its outline. They would include disallowances of certain implausible arguments, indications that arguments as presently outlined were deficient in certain respects or needed supplementation, and requirements that students concede certain points. The directions hearing was scheduled for thirty minutes. The judge explained the draft directions, and, if necessary modified them. Teams then had an opportunity to ask questions on the directions, and to seek any directions or clarification that they required.

We have mentioned the potential of contextual manipulation for learning. A partial objective of the divided hearings was to replicate a professional context, in which an expert (a partner, or counsel) gave feedback concerning the argument. Although team members can give each other “lateral” feedback of value, students must learn how to obtain, reflect on, and use “vertical” feedback, and how to ask questions to get the feedback that they need. Although there may be objections to the authenticity of a judge giving feedback, students in a moot rarely forget that “the judge” is in fact a teacher and a marker.

E. Written Submission

The traditional moot underplays the development of other skills, in favour of oral advocacy. One way to overcome this is for a team’s, or a student’s, mark to specifically include an allocation for a written submission. This requires students to demonstrate legal writing facility. Even in the traditional moot, students frequently are required to lodge something written prior to the moot, whether it be a list of cases, an outline of the submission, or the full text of the submission. The usefulness of such a requirement depends on several factors. First, if no marks are allocated to the written requirement, less effort will be spent on it. This does not motivate the development of written skills. Even if marks are assigned to it, this suffers from the usual problems of summative assessment. Second, unless the written requirement is required sufficiently early in the proceedings, it will serve little purpose other than to permit the judge to follow the argument, like the way an opera fan follows a libretto. Good judging and assessing requires the judge to have a reasonable period to prepare for the moot on the basis of the submission, in which to satisfy herself of the argument pursued, and to set questions on the basis of it. These problems decrease the extent to which a written submission can be used to present a more detailed and complex case in the proceedings.

A related problem is that the traditional moot creates conflicts for both students and judges as to how to allocate time between submission, questioning by the judge, and rebuttal of the opposition’s case.57 Often, one side’s argument will be harder than the other’s, and will require greater effort to establish it. Yet that argument will frequently be more vulnerable to attack by an acute judge. Unless one sets a long period in which to speak, there will be a conflict between the presentation of a submission and the ability to advocate it effectively by response to questioning by the judge.

The formative moot resolves this conflict by eliminating the oral submission of argument. Since the amount of preparation a judge does for the moot improves its quality, and because preparation requires knowledge of the submission, the most convenient method for resolution of the conflict is specialisation, and eliminating duplication. The written submission is delivered a reasonable time before the main hearing. In the implemented formative moot, we directed the submission not to exceed 4000 words, and that it be submitted a week before the main hearing. The onus is therefore on the judge to be familiar with the submission. This eliminates the need for duplication by orally presenting what has already been read once. The main hearing can therefore be spent entirely in the examination of that submission, and the students’ understanding of it, through a questioning procedure. In the implemented moot, each student was examined for fifteen minutes. One may object that the written submission alleviates the need for students to learn how to present an argument.58 The objection is not well founded. In order to answer a question on a submission properly students must learn to extrapolate from various parts of that submission. Thus, students do (or should) end up presenting their argument, but in a manner which requires them to reformulate it in response to the exigencies of the question confronting them. This is a more demanding variety of advocacy for the student.

One may note the interesting result which this procedure reaches. For a procedure to give students flexibility in how they learn through formative assessment techniques, much greater weight is thrown back onto the judge. The judge has a responsibility to ensure that good questions are set in order to gauge student understanding of the law. In effect, one accomplishes a reversal of the status quo of the traditional moot: it is the judge’s responsibility to understand the students, rather than the student’s responsibility to see that the judge understands them. This is a parallel to the point that we made earlier that teachers must take care with learning activities, in order to establish that they correspond with learning objectives.

F. Rebuttal

A hallmark of good advocacy is the ability to demonstrate the weaknesses of an opponent’s case. The traditional moot is not well suited to this. This is primarily attributable to the time problem of the conflicting demands of presentation and advocacy through questioning referred to in the last section. This problem is intensified for the student by the rebuttal requirement. If a student has a limited period of time, an unknown quantity of which will be occupied by questions, a student is at a loss to know how much to say in response to the opponent’s case. Additionally all of the rebuttal has to be done impromptu, since the exchange of submissions is not a conventional procedure in traditional moots.

The formative moot negotiates this problem in two ways. One has already been referred to. That is, the comprehension of the other side’s case gained at the directions hearing increases the responsiveness of each team’s submission. A second means to negotiate this problem is to formalise the rebuttal process. In the implemented formative moot, each team was required to exchange submissions when they were handed in, a week before the main hearing. The teams then had to deliver a case in reply one or two days before the main hearing. The rebuttal was thus formalised in the same way as a main submission, permitting a more detailed and reasoned rebuttal than the usual one-liner, superficial type responses often found in moots. One student in the team was responsible for defending that reply at the main hearing by replying to questions from the judge. The logic of this process is twofold: it communicates to students that rebuttal skills are important parts of the exercise; and it provides means by which student achievement in this area (facilitated through the multiple hearings concept) can be more precisely examined.

G. Developing Group Skills

We have argued in this section that because moots require a number of skills, the process can be reformulated to improve student learning in each of them, rather than primarily in advocacy We have shown how the formative moot seeks to facilitate student learning of advocacy, legal and strategic skills. We also indicated that the moot can be an occasion for students to learn group skills. Earlier in Part 11, it was noted that group skills are more representative of the “real world, which need not be limited to legal practice. The greater demands of the formative moot suggest that it would be better handled by three persons, rather than the two person teams customary in the authors’ experience.

The multi-task requirements of the moot also require considerable abilities of teamwork and co-ordination in order to be able to complete the multiple requirements of the moot59 in a competitive fashion, a demand exacerbated by the fad that teams tended to be larger than in traditional moots. The larger the team, the greater the demands on team co-ordination. Consistently with the need to assess skills alleged to be important, instructors should allocate marks to group work, such as preparing the written submission.

The next part describes the implemented procedures in more detail, and the results of surveys of the moot’s effectiveness.

IV. THE FORMATIVE MOOT IN ACTION

A. Procedure and Setting

We have noted that the factual problems were hypothetical cases involving the assessability of the proceeds of transactions either as income under s 25 of the Income Tax Assessment Act, or under the capital gains provisions in Pt IIIA.60 We have noted that students were allocated to teams of three.61 Students were required either to act for the taxpayer or for the Commissioner of Taxation in an appeal by the taxpayer to a single judge of the Federal Court against an assessment under these provisions. The time frame of the Moot proceeded along the following lines (with minor variations):

Students were encouraged to assign two members of the team to defend the substantive arguments in the submission, and one to defend the case in reply. In order not to bias survey or follow-up results, all moots were judged by one of the authors sitting alone. Students were informed that they would be marked on the following matters:

• Comprehensibility and clarity of Outline of Argument
  
  (5 marks)
• Intelligent or strategic use of Directions Hearing — (5 marks)
• Overall appearance of co-ordination in group performance
  
  (5 marks)
• Quality of legal argument in Submission of Case and Case in reply— (35 marks)
• Observance of court etiquette— (5 marks)
• Clarity and confidence in addressing the court— (5 marks)
• Quality of advocacy— (40 marks)

Components (a)-(d) were marked for each team, components (e)-(g) were marked for each individual. Thus, group and individual marks were divided equally. The criteria reflected the skills that the moot sought to develop: legal research, writing, teamwork, strategy and oral advocacy. After moots were completed, students were given their mark (broken down as above), and written feedback on their group and individual performance.

B.  Results

Data was collected in two ways. First, at the end of each moot, students were asked to complete and return a survey questionnaire. The questionnaire contained open-ended questions asking for written responses, and multiple choice questions. Of the 104 students completing a moot, 100 students returned their questionnaires. Second, after the perusal of these results, we further investigated issues emerging from them in two focus group sessions with an average of five students attending. These focus group sessions generated further qualitative evidence on student perceptions of the learning process.

1. Response to Formative Assessment

The formative moot procedure increases the demands of students considerably in excess of what a traditional moot requires of them. One student described the three week procedure in the survey questionnaire as “very exhausting”, a view generally shared. Nonetheless, 79 per cent of student respondents to the questionnaire considered that the allocation of 40 per cent of the marks for the subject to the moot was a fair one, given its demands.62 Likewise the student quoted above said, in the same phrase, that the moot was “very rewarding”. Students also did not perceive the longer period of the moot to be unnecessarily protracted. Seventy nine per cent of students considered that period to be about right.63

A number of the key findings on a formative approach to moots are discussed below in connection with the directions hearing. However, several students in questionnaires appreciated the staged development of their participation in the moot, each building on their learning in the previous stage. Although feedback was not specifically sought on this point, some students found the development of a package of written documents — the outline, the submission and the reply — to be satisfying. This theme came through in the focus group, one student stating his opinion that “[t]he stages were the best idea, given that they defined everything and allowed for filtering out of irrelevant material and arguments. Th[at] process is much more efficient.”

Also, some students made unsolicited comments about the procedure having a greater feeling of verisimilitude about it. This feeling was summed up by one student who wrote in the questionnaire: “[I] [felt as though I was participating in real litigation; [there was a] real sense of progression from beginning to conclusion of case.” The notion that students felt they were learning from something discernibly “authentic” is consistent with cognitive apprenticeship theory. From our perspective as judges, students seemed to feel more confident about their advocacy in their written submissions and oral advocacy, because they felt confident that their arguments were plausible, given the existence of formative feedback. We pursued this point further in the focus group, when we asked students whether they thought that, if given the same problem as a hypothetical assignment, they would have answered it as proficiently as they analysed it in their written submissions. Everyone thought that their moot submission would have been better, and that they learnt more from the moot. This perception was attributable to the formative feedback, and also to the ability to interchange ideas in a group situation. The moot’s learning objective to encourage deeper learning seems to have been achieved.

2. The Directions Hearing Concept

As noted above, the directions hearing was perhaps the most important phase for the provision of formative feedback to students. Additionally, it provided an opportunity for students to influence their learning by means of seeking directions for the form of their submissions and the conduct of the main hearing. The latter opportunity was underutilised by students. When asked to describe the purpose of the directions hearing in the questionnaire, students overwhelmingly emphasised the first function. They generally stated that the hearing was a process for ascertaining whether they were “on the right track (a phrase often used), and to eliminate lame or unhelpful submissions. Thus, the formative assessment function was understood, and appreciated. Students were asked whether they thought the directions hearing achieved the purpose that they had described. Ninety-two per cent of respondents thought it did. While such a response is difficult to interpret because students differed somewhat as to the purpose they understood to prevail, the formative feedback purpose was widely shared.

It is important to reflect on underutilisation of the moot’s strategic possibilities. As a generality, better strategic use of the moots was made by later groups than earlier groups. Two factors were significant here. The first is that students were required to complete this moot at a very early stage of the semester. The earliest groups collected their topics after just two lectures on income tax, one on tax policy, the other an elementary outline of key concepts of income tax and capital gains. For such groups to come to grips with the law with such a small knowledge base was a very considerable effort. Only 35 per cent of students competing in the first ten (of eighteen) moots thought they had learnt enough about tax to permit them to conduct research for the Moot. Unquestionably, this was intentional, as it greatly accelerated independent, and deep, learning.64 However, this comes at the cost of students feeling somewhat less assured about the law, which decreases their ability to take strategic action. This point is intensified by the comparatively early stage of the directions hearing in the procedure — six working days after topics were collected.

The second factor influencing the greater proficiency of later teams is the dissemination within the student body of understanding and learning about the directions hearing concept itself. The hearing is new to student mooting. Thus, students only gradually came to recognise the opportunities the hearing presents, and plan their own participation in it. Both factors were recognised in focus group sessions. Generally, students in later teams did comment in their questionnaires that they saw that the directions hearing did present these opportunities,65 and regretted their failure to take them up. Some students felt that their ability to respond strategically might have been facilitated if they had the draft directions available before the directions hearing. This would however either lengthen the process, or require earlier submission of the Outline.

The directions hearing should provide an opportunity for each team to learn about the other’s submissions. This, together with the judge’s formative feedback, enables the teams to write submissions that are more responsive to each other (so adding authenticity to the learning activity).66 The data suggests that this function of the hearing was a success. Seventy three per cent of survey respondents considered that after the directions hearing, they had a reasonable idea of their opponent’s submissions. The assistance given by formative feedback to student learning was even more successful. A weighty 98 per cent of respondents considered that the directions hearing assisted them in writing their submission.

Students were asked whether it would be better to replace the directions hearing with written directions only. Only 10 per cent thought so. Of the 90 per cent who disagreed, reasons varied. The main reasons why the hearing concept was preferred were, first, the ability to ask questions (and thus generate further formative feedback). Second, a number commented that the directions hearing served as a sort of “ice-breaker” which provided familiarity with the judge and improved oral skills. This suggests a certain degree of learning by self reflection on one’s experiences.

The directions hearing therefore seems to be a valuable and appreciated means of improving the quality of student learning, through its formative feedback. It also provides an opportunity for a form of strategic advocacy. Our evidence suggests that this form of strategic advocacy is most likely to make a positive contribution to learning outcomes where students have a secure grounding in the substantive law, and understand the sorts of directions that they might seek.

3. The Written Submission

The written submission is a familiar concept in some traditional moots. However, a formative moot changes its role and process substantially. First, the structure and content of the submission will usually be influenced by the directions hearing. Directions should improve the efficiency of the research and writing for the submission. Our evidence indicates that directions do serve this purpose. Second, the formative moot substitutes the written submission for oral presentation of the argument, in order to spend court time in the main hearing examining students on the submission. The written submission thus ceases to be a transcript of a prepared speech, and becomes the formal legal argument. We suggested that a problem with oral submissions is that the complexity and detail of submissions are compromised by the competing demands of questions from the judge and rebuttal duties. Although most assessors will wish to set a word limit on the submission, the directions hearing enables the elimination of matters that should be conceded and other unproductive submissions, so permitting the word limit to be used more productively by students.

In general, students appreciated the opportunity to present a written submission of their arguments. Eighty six per cent of students preferred to deliver written submissions to oral submissions. Sixty six per cent of students considered that written submissions provided a greater opportunity for the student to demonstrate ability in legal research. Some of the preferences for the written submission derived from aversion to public speaking. Nonetheless, the formative moot requirement for the student to answer questions without notice at the main hearing can hardly be a “soft” option for weak orators. One student encapsulated the matter thus:

There are a number of people who loathe public speaking and for whom such a task, combined with the need to explain the intricacies of a complicated argument, is an ordeal. However, with some time everyone should be able to prepare a written submission that will reflect the effort they have expended. In an oral presentation, nerves and/or one or two difficult questions can floor people, obscuring the actual time [and] effort put into research and the knowledge they may have of the issues.

Thus, the dislike for public speaking can be seen as a manifestation of the compromise between submission and questioning, and the distorted feedback concept, that are inherent in the traditional moot. Presenting an oral argument about a technical area of law such as income tax, which relies on complex written provisions, is a difficult job even for a professional advocate. In a similar way written argument is important for tax for another reason: much of the legal process in revenue matters is written. Objections to assessments, and objection decisions are all in writing. The Commissioner makes written rulings. The case for written submissions in this area is very strong.

A number of students commented favourably on the opportunity to demonstrate legal research and writing skills. One student observed that “[a] written submission is more efficient. It makes you structure your argument. ... It gives you the chance for comprehensive research.” Others also commented on greater opportunities for clarity and comprehensiveness. On balance, the written submission served the purpose we had hoped for.

4. The Rebuttal Requirement

The ability to respond to and criticise an opponent’s submission is an integral part of advocacy. As we have argued, the traditional moot compromises it, to an extent. Unless each student is allocated a right of reply time specifically, it creates a further conflict for the student ascertaining how to allocate his or her time for address. Rebuttal is thus performed somewhat superficially. This aggravates the inability of the traditional moot to bring opponents to issue. Should rebuttal be presented orally, or in writing? In a pilot study for this moot project, students responsible for oral rebuttal often stated, perhaps untruthfully, that the questions the judge had put to their opponents pre-empted their line of discourse. It follows that each team should have an opportunity to put a case in reply as an extension of its own written submission. This, in turn, increased the standardisation of assessment of students responsible for rebuttal with those who were not. A majority of those responsible for rebuttal (52 per cent) thought that their job was as difficult as the other team members’ jobs, although a significant minority (33 per cent) thought it was harder. These students stressed that they felt obliged to have the same knowledge as the two other team members in order to perform their job.

As with the written submission, students found the written rebuttal to be an efficient means of delivering a reply. Fifty nine per cent preferred a written submission, and a further 16 per cent would have preferred to use written and oral submissions.67 Some students commented that the knowledge that the other team would be receiving their submission and preparing a reply on the basis of it, increased the care with which the submission was written. Thus, the formalisation of rebuttal tended to increase students’ critical self-reflection.

More generally, the formative moot process brings both sides together at an earlier stage. This influences the final products (the submission and the reply), because it increases the extent to which these respond to each other. Thus, one student observed that, “[w]e were aware they [ie their arguments] might change. So we used their directions to assist us in what they would argue and how they would do it.”

5.  Examination in Main Hearings

The formative moot uses the written submission to address the conflict between submission and questioning. All main hearing time is spent in questioning students on their submissions, with an average period of about 15 minutes per student.68 The procedure is not unlike the assessment of material viva voce, infrequently observed in Australian law schools today. The preparation of questions is the assessor’s most demanding and time consuming task. If questions are not well set, the student will find it difficult to demonstrate understanding of the material in a way that advances his or her team’s case.

Despite some complaints about the complexity or the fairness of the questions,69 students generally reacted very positively and in intended ways to the questioning procedure. First, students found the relationship between the written submission and extended oral examination to be a positive one. A number commented in the questionnaire that the written submission gave them an opportunity to develop the case to the degree of detail that they liked, and the oral procedure allowed them to clarify any ambiguities, or expand on more contentious points, in that submission.

Second, students generally found that the written submission improved their learning, and gave them an opportunity to demonstrate that learning. A typical comment in the questionnaires was that the procedure “requires you to do more than just rote learn and be prepared to answer the everyday standard questions that are asked in moots.” The perception of the traditional moot as fostering “regurgitation” was common amongst respondents. The time that students often spend in the memorisation of oral address in a traditional moot was rechannelled into closer study of the law. One student stated in a questionnaire, “Not having to give submissions orally took off a lot of pressure and allowed us to concentrate on knowing arguments and cases.” Also, some students find the recitation of an argument in the traditional moot to be a bore. Boring procedures, as part of context, are unlikely to engage the attentions of students. The same students found the questioning procedure more intense and interesting.

Third, the questioning procedure clearly exerted a motivating effect. When asked, “Do you think that the knowledge that you were to be questioned on your argument, made you work harder in preparing for the Main Hearing?”, 59 per cent of students considered it made them work much harder, and a further 30 per cent thought it made them work harder to some extent. Thus, one student observed in the questionnaire that “[t]he fact of not knowing what questions could be asked made you rely on knowing every aspect of your argument.”

Fourth, as with the rebuttal requirement, students found that the knowledge that they were to be questioned made them reflect more on the content of their submissions. One student said in a focus group that questioning “forces you to look at the weaknesses of your argument and as such means that when writing the submission, you needed to take into account potential questions that could be asked.” Again, this means that students reflect on their own submissions more critically, and take a greater responsibility for their own learning. The dynamics of groups also provided impetus for this critical reflection.

This procedure is not free of problems. One of the key problems with the procedure is that it is inherently negative. As a judge, one inevitably focuses on more problematic aspects of the submission, in order to test student understanding of the law and the limits of their argument. Thus, one is inclined to concentrate on finer, problematic points, often in a way that may place disproportionate weight on one part of the submission. A not uncommon response in the questionnaire was that questions “seemed to be concentrated on a narrow section of each submission”. Some students felt that this tended to prevent them from displaying their knowledge, which was thus a disadvantage compared to oral submissions. On the other hand, the negativity of questioning must be balanced by students’ greater assurances that they were considering the right areas, permitted by the directions hearing. On balance, quantitative evidence from the questionnaires seemed to show that the questions asked worked well. Sixty-one per cent of the respondents thought them highly relevant, 37 per cent thought them somewhat so. Only 8 per cent found questions too difficult, with 91 per cent finding them either somewhat difficult or about the right level of difficulty, the latter responses being the desired level. Only 18 per cent of respondents considered questions to be somewhat unfair, the remainder of the class finding them either quite fair or fair in all respects.

Some students found that requiring a written submission a week prior to the main hearing prevented them from correcting their mistakes or adducing new material. While true, the latter phenomenon can be said to be a practice discipline for the settling of “real world” documents such as pleadings.

6. Learning Outcomes and Skills Development

Comments have already been made that students perceived the procedure to be a more realistic one, both procedurally and in the demands it placed on them. Students felt that their legal and advocacy skills were being developed. Quantitative evidence is consistent with the ability of the formative moot to develop skills, and its superiority to the traditional summative moot. Although students had done two moots before, the second being as demanding as any traditional moot, 92 per cent thought that the moot improved their advocacy skills.

Although moots are normally regarded as facilitating the development of skills, rather than substantive law abilities, 95 per cent of students said that their knowledge of tax law had increased either to a significant or a major extent in consequence of the moot. One student observed that the moot

broadened the scope of legal advocacy learning. If not for assessment items such as this, I personally wouldn’t scour in the legislation, nor read the explanatory memoranda and rulings. It really helps to improve your motivation for a subject when emphasis is placed on individual responsibility rather than spoon feeding.

Here, we find a direct confirmation of the intended learning objective — using the moot as a means of motivating deep learning about tax law, by reference to its primary sources, rather than drumming through lecture notes. This effect is particularly important, given the need stated earlier to avoid superficiality of treatment of such a massive area. Equally pleasing was the fact that 71 per cent of respondents found that they were more interested in the subject after the moot. In open ended questions, a number of students frequently used the word “enjoy”, which is not conventionally associated with moots, much less income tax!

The imprimatur given to the overall concept of the formative moot was surprisingly high. Despite the greater demands the moot imposed, and other problems with teamwork (discussed below), 81 per cent of students thought this mooting procedure was preferable to others they had encountered. One student describes the learning experience in the following terms:

I gained a greater understanding of revenue law as I was forced to research and comprehend the law as opposed to merely doing a little reading and relying on lecture notes to get me through. Furthermore, the moot process forced me to think and actually extend myself. I discovered (shock, horror) that revenue law can be relevant and interesting.

7. Group Work

The major problem in this mooting exercise was group work. In response to an open ended question asking students how the moot could be improved, problems with group work emerged more frequently than any other issue. The ability to work in groups is an important skill which students need to acquire. Accordingly, the implemented formative moot placed insistent demands on group abilities. Students were required to produce three documents in a three week period, totalling around seven thousand words, in a complex area of law. Fifty per cent of marks were allocated to group performance, which included documentary outputs. Moreover, the three person group was the norm, which contrasts to the more usual two. While the use of a three person group increases the potential output of the group, it increases difficulties of co-ordination. The rebuttal function added a new complexity to decisions about work allocation and delegation within the group. Rebuttal could only begin a week prior to the main hearing when the submission was due. Teams therefore had to plan demands on team members across time, in circumstances where the difficulty of the relevant task was uncertain.

Under these circumstances, deciding a method for group formation is difficult. In the circumstances, we chose to allocate students to teams, and rejected a voluntary formation method. The key criterion for allocation was discriminatory — we sought to equalise levels of ability between groups, not within them. The motivations for doing this was twofold. One was to ensure a reasonably even contest between teams. The second was a belief that group skills would be more greatly extended if students had no choice about those with whom they worked. Hitherto unconstituted teams would need to develop (rapidly) means for making decisions about dividing duties and pooling results. The workplace often presents such demands, and we sought to replicate this situation.

Several teams experienced shirking problems. Out of 36 teams, the respondents in 26 of them answered unanimously that there had been no shirking. Of the ten problem teams, shirking perceptions varied. In four teams, only one person reported shirking, and often explained that while it was present, the extent was by no means major or extreme. Of the remaining six cases, interpersonal relations in one team seemed to break down completely, while the other teams involved both single shirker and double shirker situations. While some problems in 28 per cent of teams, and serious problems in 17 per cent of situations is a concern, we doubt that voluntary team allocation would actually have changed the situation — it may simply have shifted it to another group that was more forgiving of a lazy friend. Alternatively, some shirkers may have been left out of group formation, and assigned to teams composed of the unfortunate students who nobody wanted. The problem seems not to be so much one of groups, as one of unmotivated students. One student who complained of being saddled with two shirkers made the following revealing comment:

I learnt a lot about group dynamics, probably more so than S 25 [and] S 160M(7)! I know exactly how I would approach another group assignment and what I would do differently if I had to do it all again tomorrow. I really enjoyed the whole experience, despite the long lonely hours of work and sleepless nights.

For all the student’s anguish, here is a classic case of skill being developed — how to handle group problems. Many of the students who had unfortunate group experiences complained about the heavy weighting of group marks, and thought it would be better if individual marks were increased. However, this needs to be set against the reverse problem. Many complained that for such major undertakings as the outline, submission and reply, the allocation of as many marks as for the fifteen minute main hearing appearance was disproportionate. Thus, it is largely a question of balance, about which student views will depend on their group experience.

For the 70 per cent of groups who had no reported shirking problems, the group experience often was a positive source of reward, sometimes of enjoyment. Some people enjoyed the challenge of meeting new people, or working with others whom they would not have chosen had they had the liberty to choose. A number of students found that working in groups enhanced their ability to think more critically about the arguments. Because students were marked on the extent of group co-ordination, and were specifically warned about handing in submissions which concatenated arguments having nothing in common, it was important to synthesise and edit research findings. This supplied the opportunity for critical reflection. Overall, despite inevitable reservations, the experience was in our opinion a valuable one for the development of interpersonal skills. This is perhaps best exemplified in two student comments. One student said:

I usually despise group work because of my selfishness and free-rider problems. I believe this moot taught me to be a little more patient and understanding of the opinions of others.

Another said:

I got to know my teammates well although I had little to do with them before. It helped me develop qualities of tolerance and appreciation when things were a bit difficult eg tolerance when a team member was late or unable to turn up, but appreciation when they went out of their way to do extra work such as photocopying or hunting out cases. The moot seemed to unite all the third year [students] by providing a common focus or talking point.

Patience, understanding, tolerance and appreciation are surely “skills” law graduates could do with. After all, many law graduates expect that these virtues be demonstrated to them.

C. An Appraisal

Our theoretical analysis has sketched an unflattering image of the traditional moot. This has been confirmed by our empirical evidence, and by the unsolicited comments of some students regarding the traditional moot as “boring” and “unrealistic”. This picture understates its virtues. Even a flawed means for developing advocacy skills will still have some value for almost all students, at least as a heuristic exercise. Its longevity as a form of assessment should give cause to reconsider dismissing its merits.

It needs to be pointed out that the formative moot creates considerable resource and time demands. The judge can expect to spend between five to six hours per moot, excluding the time it takes to set questions and to mark, but including two hours of court time. More time is spent reading outlines, submissions and replies, drafting directions, and preparing questions, than is spent in the courtroom. This figure also includes such economies as flow from doing up to five moots on the same question. Therefore, while the rewards of the formative moot are high, they come at a cost. In particular, they require discipline and speed from the judge, especially with requirements like a 24 hour turnaround of directions for five different moots. In this era of rising workloads, higher expectations, and continuing lack of reinforcement of good teaching, many may judge such a procedure to be too unattractive and burdensome.

In light of those costs and demands, it is impossible to plead for the abolition of the traditional moot. Instead, we believe our analysis needs to be considered by law teachers and those responsible for skills development and training. Our project suggests a need to think about moots generally.

First, the formative moot lends itself well as an assessment item for (i) later year subjects; and (ii) subjects (especially electives) with smaller enrolments, or with a larger teaching team. If students are already familiar with traditional moots, they are more likely to understand (and, hopefully, to appreciate) its procedural variations. The formative moot has considerable potential as a technique for teaching both substantive law and a range of skills, and may be applied within both substantive law and general moot subjects. The comments in relation to smaller enrolments or larger teaching teams respond to resource considerations. The notion that the moot is best deployed in later year subjects acts as a control on the need to ensure that students have reasonably well developed legal research and writing skills. These skills are vitally necessary for the formative moot, and need to be applied under strict time constraints. This may be counterproductive for first year students.

Second, the formative moot can be thought of as a portfolio of techniques, which can be selectively employed by a teacher to manipulate the context of skills activities. The directions hearing can be used alone. It is the key formative assessment element in the moot. It could be used even if submissions are delivered orally It would enable a judge to correct major problems in outlined arguments, to clarify concessions, and to give directions that students seek on facts. Such a hearing would also serve to facilitate the judge’s understanding of oral submissions, and may permit these to be somewhat shorter. The written submission/oral examination procedure could also be used alone, or modified to have much shorter oral presentations of argument. Either approach indicates more clearly to students how the conflict between presentation and submission will be resolved by the judge. This enables students to prepare for the main hearing with more realistic expectations. The formalisation of rebuttal is also a tactic that can be used in conjunction with written submissions handed in ahead of time. A written submission and some oral presentation would give an opportunity for students both to respond, with more deliberation and consideration, to the overall case documented in the submission, as well as to prepare material to discuss in rebuttal at the hearing, based on new issues arising from the opponent’s oral advocacy. Making a third student responsible for rebuttal will help this process, by removing the conflict between presentation of one’s own case, and rebuttal of the opponent’s. These suggestions show how moots can be improved incrementally and the contribution of each technique studied in isolation.

Third, our analysis suggests that law teachers need to consider how they use moots to develop skills. All moots require legal research and analysis, legal writing and team work, as well as the obvious advocacy requirements. It is important that law teachers should take the opportunity to see that the skills are developed properly. The simplest way to do this is for the marks allocated to the moot to be divided into categories that correspond to the necessary skills, and for feedback to be given which addresses the level of achievement in these skills. It is important, however, that the instructor be careful to ensure that the skills assessed are in fact developed. There needs to be some positive input into these skills, not just passive, and summative, measurement.

Fourth, this project has implications beyond mooting. It shows clearly that students do respond to formative assessment techniques, as well as techniques that stress understanding and deeper learning. In relation to the former, it is scarcely possible to overstate the need to consider the formalisation of the provision of formative feedback to students during the course of assessment items. In our project, students seemed to learn more and the quality of their work was higher. Formalisation encourages organisation and teaches good work habits. The need to stress understanding is hardly a novel insight. Our comment will be limited to the observation that, in this project, understanding arose from a procedure that was plainly onerous for the students completing it. Nonetheless, difficulty counts for little by itself, unless it is allied to careful identification of the skills that such difficult assignments are intended to develop, and the use of formative assessment techniques. Both the identification of skills and the nature of formative assessment must arise out of the subject learning objectives. Traditional moots are always perceived as difficult undertakings, but the student who arrives for a hearing with a view of the case that differs from the judge’s may learn very little, no matter how arduous the ordeal. Such difficulties may only develop skills students are not intended to be acquiring — such as the ability to bluff and bluster one’s way through an oral submission.

V.  CONCLUSIONS

In the 1990s, the expectations of legal education are higher than ever. The cost of a law degree to an undergraduate student has grown considerably while the funding of many law schools has fallen in real terms. Students justifiably want more for their dollar, but the resources available to deliver teaching have not increased. At the same time, constituencies within and outside the academy have criticised the content and delivery of legal education. It is a stimulating, but sometimes uncomfortable, time to be a law teacher. Skills training has been a common element of all three of the themes just described — satisfying student demands for a relevant, useable degree; efficient resource utilisation; and critiques of legal educational method.

It was argued above that the theory of skills training has been left behind by its practical implementation. Anecdotally one can detect this in the eclectic catalogue that academics are told to believe is the universe of relevant skills. Students apparently should learn advocacy but no mention is made of Bayesian probability theory as a means of analysing the likelihood of convincing a tribunal of fact of some conclusion.70 Students should learn negotiation skills, but no mention is made of non-cooperative game theory as a means by which to model strategic interactions.71 “Theory”, whether Bayesian or game, is not perceived, or treated, as if it had any connection with formal skills. Rather they are presented, if at all, as doctrinal critiques, much as one would teach feminism or postmodernism. However, theories such as these are highly relevant to the durability of the skills we impart to our students. As technology and competition transfigure the legal profession, choosing the skills in which to train students according to the criterion that lawyers are believed to use them will become an increasingly poor strategy. This skills “positivism” will be as deficient in terms of its normative implications for legal education, as legal positivism is for law reform policy.

In light of our analysis, two points can be made. The first is that teachers and law schools must spend more time thinking about learning objectives, and the implications of these for skills training and assessment, subject to resource constraints. There are no easy answers here, and the eventual decisions will owe as much to law school politics and external influences as they will to educational theory. The second is that when teachers decide to teach particular skills, they should not assume the appropriateness of traditional methods for delivery. We have shown, by reference to moots, that there are skills which some teaching activities could develop, but fail to emphasise, and that skills which are emphasised may be compromised by the parameters of the activity. The practical implication is that controlled experimentation with existing forms has much to recommend it. Above all, teachers must look carefully at the opportunities for providing formative feedback, and assessing the impact it makes on student learning. Unless the work of law teaching is motivated by improving student learning, its onerous demands are hardly worth the effort.

* BA LLB (Hons) (Qld) Grad Cert Higher Ed (Griffith), Griffith University.

** BCom (Hons) LLB (Hons) MFM (Qld), Griffith University. We wish to thank the Griffith Institute for Higher Education for its generous funding and encouragement of this project, Dr Peter Taylor and Professor Royce Sadler for their valuable comments and input, and Oliver Bennett, Rebekah Fryer and James Smith for research assistance.

©1997 [1997] LegEdRev 1; (1997) 8 Legal Educ Rev 1.

1 J Heywood, Assessment in Higher Education (London: Wiley, 1977) vii.

2 From students’ perspectives, “assessment always defines the actual curriculum”: P Ramsden, Learning to Teach in Higher Education (London: Routledge, 1992) 187.

3 P Candy, Self-Direction for Life Long Learning: A Comprehensive Guide to Theory and Practice (San Francisco: Jossey-Bass, 1991) 251. See also R Mayer, Cognition and Instruction: Their Historic Meeting within Educational Psychology (1992) 84 J Educ Psychol 405.

4 Ramsden, supra note 2, at ch 5.

5 R Riding, & I Cheema, Cognitive Styles: An Overview and Integration (1991) 11 Educ Psychol 193; R Johnstone, Rethinking the Teaching of Law [1992] LegEdRev 2; (1992) 3 Legal Educ Rev 17, at 31–32. Some learning theorists appear to make the crude assumption that the teacher may manipulate the learning context to achieve the same outcome for all the members of any group of students. That assumption cannot be reconciled with the literature on cognitive styles.

6 D Kolb, Experiential Learning: Experience as the Source of Learning and Development (New Jersey: Prentice Hall, 1984); D Boud, R Keogh, & D Walker, Reflections: Turning Experience into Learning (London: Kogan Page, 1985).

7 Schon has observed that critical reflection is an important skill possessed by expert practitioners. His argument that critical reflection should be one of the most important values in professional education has been influential in legal education and other disciplines: D Schon, The Reflective Practitioner (Aldershot, Hants: Arena, 1996); D Schon, Educating the Reflective Practitioner: Towards a New Design for Teaching and Learning in the Professions (San Francisco: Jossey-Bass, 1987); M Le Brun, & R Johnstone, The Quiet (R)evolution: Improving Student Learning in Law (Sydney: Law Book, 1994).

8 The use of formative assessment, and the provision of feedback to students are examples of the way in which teachers may attempt to encourage and develop students’ abilities in self reflection.

9 American Bar Association, Legal Education and Professional Development — An Educational Continuum, Report of the Task Force on Law Schools and the Profession: Narrowing the Gap (MacCrate Report) (Chicago: ABA, 1992), assumes the validity of experientialism as the basis of skills education in the university. Cf J Costonis, The MacCrate Report: Of Loaves, Fishes, and the Future of American Legal Education (1993) 43 J Legal Educ 157, at 171–5, 180, 190–1. Costonis queries whether the “pale” simulations of professional experience which law schools can provide is an adequate basis for experientialism: at 175.

10 National Board of Employment, Education and Training, Developing Lifelong Learners Through Undergraduate Education (Canberra: AGPS, 1994) chs 2–4.

11 Given the rapid rate of change in the content of, and procedures employed in, any discipline, the argument is that what may be meaning fully taught will be transferable: id; C McInnis, & S Marginson, Australian Law Schools After the 1987 Pearce Report (Canberra: AGPS, 1994) 25.

12 Schon, Educating the Reflective Practitioner, supra note 7, at ch 11.

13 J Seely Brown, A Collins, & P Duguid, Situated Cognition and the Culture of Learning (1989) 19:l Educ Researcher 32; L Resnick, Learning in School and Out (1987) 16:9 Educ Researcher 13; A Collins, J Brown, & S Newman, Cognitive Apprenticeship: Teaching the Craft of Reading, Writing and Mathematics, in L Resnick ed, Knowing, Learning and Instruction: Essays in Honour of Robert Glaser (New Jersey: Erlbaum, 1989). Cognitive apprenticeship assumes that identifying the relevant culture and its authentic activities are unproblematic. This theory has the potential to uncritically reify current practice.

14 Seely Brown et a1 base their argument on the assumption that craft apprenticeship is “evidently successful”: supra note 13, at 37. Given the serious claims made for apprenticeship over other types of learning, their failure to explain craft apprenticeship and the basis on which they assume its validity is problematic. In legal education, apprenticeship based education is regarded as impoverished and inadequate: see eg J Thomson, Objectives of Legal Education — An Alternative Approach (1978) 52 Austl LJ 83, at 83, 85; R Posner, The Problems of Jurisprudence (Cambridge: Harvard, 1992) 2–3. However, Seely Brown et a1 state, as evidence that “apprenticeship” may be an effective method of learning both cognitive as well as practical material, that “many professions with generally acknowledged cognitive content, such as law ... have nonetheless traditionally been learned through apprenticeship”: 39. It is simplistic to endorse cognitive apprenticeship in legal education without a thorough consideration of the issues involved. For instance, this theory also fails to recognise the economic and social reasons (eg monopoly explanations) for apprenticeship and guild-like organisation: R Posner, Overcoming Law (Cambridge: Harvard, 1995) 37–44 (comparing legal profession to medieval guilds).

15 We have reservations about the application of cognitive apprenticeship to law, including the difficulties of replicating authentic activities, establishing how students recognise activities as authentic, and what it is that students enrolled in a LLB degree are apprenticing for. Cf Johnstone, supra note 5, at 38–42, Le Brun & Johnstone, supra note 5. Laurillard provides an interpretation of cognitive apprenticeship which explains its contribution to understanding the role of teachers in mediating learning: Rethinking University Teaching: A Framework for the Effective Use of Educational Technology (London: Routledge, 1995) ch 1.

16 Le Brun, & Johnstone, supra note 6, at 180. The majority of the literature on assessment is cast in terms of practical advice to teachers engaged in subject design. This advice appears to be based in learning theory, although it is rare to see the bases articulated.

17 S Brown, & P Knight, Assessing Learners in Higher Education (London: Kogan Page, 1994) at 46; Ramsden, supra note 2, at 67.

18 Laurillard, supra note 15. There should ideally be progressive development of all aspects of the curriculum, based on a high degree of planning and co-operation between different levels and different subjects. However, for the purposes of this paper we will assume this is unlikely to be realised.

19 Awareness of the importance of feedback is not novel. The Committee on Curriculum of the American Association of Law Schools stated in 1945 that “the lesson of apprentice training is that it is the redoing after critique which is the crux of the learning”: The Place of Skills in Legal Education (1945) 45 Colum L Rev 345, at 373.

20 We hesitate in doing so because this overemphasises the responsibility of individual teachers. Institutional planning and responsibility in assessment is crucial, given the ongoing nature of learning.

21 D Rowntree, Assessing Students: How Shall We Know Them? 2nd ed (London: Kogan Page, 1987) 31; J Barnes, The Functions of Assessment: A Reexamination (1990–[1991] LegEdRev 10; 1991) 2 Legal Educ Rev 177 at 212–3.

22 For a comprehensive statement of learning objectives in legal education, see Johnstone, supra note 5, at 22–8.

23 Rowntree, supra note 21, at 90.

24 Barnes, supra note 21, at 180–182; Ramsden, supra note 2, at 188–189.

25 Rowntree, supra note 21, at 92.

26 Eg T Crooks, Assessing Student Performance (Campbelltown: HERDSA, 1988) 11, 22–23. This seems loosely to be based on constructivism, although teachers should avoid making assumptions about students’ ability to comprehend and respond to stated objectives and criteria. Sadler points out that these assumptions are often misplaced: R Sadler, Formative Assessment and the Design of Educational Systems (1989) 18 Instructional Sci 119, at 134–5.

27 D Boud, Assessment and the Promotion of Academic Values (1990) 151 Stud Higher Educ 101, at 103–104, Brown, & Knight, supra note 17, at 33.

28 On the distinction between formative and summative assessment, see Sadler, supra note 26, at 120 (the distinction fundamentally relates to purpose, and not necessarily to timing or placement of exercises); Rowntree, supra note 21, at 121–122; Le Brun, & Johnstone, supra note 6, at 181–182.

29 Brown, & Knight, supra note 17, at 38–41.

30 The distinction between formative and summative assessment is usually applied within the context of individual subjects. Of course, it is deceptive to assert or accept that a terminal item of assessment will have no further implications for student learning.

31 Sadler, supra note 26, at 120.

32 The focus in assessment until recently has been on its certification function, which explains the prevalence of issues of validity and reliability: Sadler, supra note 26, at 122.

33 See Brown, & Knight, supra note 17, at 38. We do not deny that in some cases, summative assessment is the only type of assessment which is practically feasible. We are not asserting that summative assessment does not affect student learning.

34 Although this is theoretically possible, it is difficult to achieve. Publication of key aspects of the subject design, particularly the assessment criteria, are intended to achieve this aim.

35 It particularly denies the significance of individual variation, contrary to the literature on cognitive styles: see supra note 5.

36 For example, Sadler notes that feedback is commonly defined as “information given to the student about the quality of performance” (emphasis added): supra note 26, at 142. The external sources of feedback include peers and teachers.

37 Sadler, supra note 26; Rowntree, supra note 21, at 27; D Boud, Implementing Student Self Assessment 2nd ed (Campbelltown: HERDSA, 1991).

38 Feedback should be both actually and appropriately used in order to justify the cost to the teacher. A key assumption relied on by many writers is that if feedback is related to stated aspects of the subject design, it will be appropriately applied by students. However, research indicates that “even when teachers provide students with valid and reliable judgments about the quality of their work, improvement does not necessarily follow”: Sadler, supra note 26, at 119, 134–5. This implies that methods of applying the feedback should be embedded in subsequent learning activities.

39 This may be done by using activities which draw on multiple skills, or by including a number of different types of activities (eg oral presentation, written paper) within the assessment schedule: Brown, & Knight, supra note 17, at 23–24. This also minimises bias and preference for particular cognitive styles. Cf supra note 5.

40 Boud, supra note 27, at 106.

41 R Reed, Group Learning in Law School (1984) 34 J Legal Educ 674. The judicious use of group work can have the incidental advantage of reducing teachers’ work load. For example, students can provide feedback to each other, which is likely to be particularly effective if they are working together on assessable work (their detailed knowledge of the area will be high, and there are obvious extrinsic motivations to providing quality feedback in order to assist colleagues in completing high quality work), and the number of items which teachers are required to assess will be less than where students are individually assessed.

42 In the USA, the inadequacies of the traditional model of legal education were addressed by broadening content, particularly by the inclusion of interdisciplinary and theoretical material, and by including “skills training”. See Costonis’ discussion of the three aspects of modern legal education: education in “technical knowledge” (substantive doctrinal coverage, including legal analysis), “general education” (“interdisciplinary, policy oriented inquiry”), and “practical training” (skills): Costonis, supra note 9, at 162–164. Developments in Australian legal education have followed this tripartite composition: C Sampford, & D Wood, Theoretical Dimensions of Legal Education — A Response to the Pearce Report (1988) 62 Austl LJ 32.

43 The interest in skills education in Australia was fuelled by comments in the so-called Pearce Report: Commonwealth Tertiary Education Commission, Australian Law Schools: A Discipline Assessment for the Commonwealth Tertiary Education Commission (Pearce Report) (Canberra: AGPS, 1987).

44 McInnis, & Marginson, supra note 11, at 168.

45 Le Brun, & Johnstone, drawing on the MacCrate Report, list 49 skills which “can form an appropriate and respectable part of teaching in universities”: supra note 7, 170–172.

46 The width of coverage in this conception of skills can lead to confusion. Different considerations underlie the use of legally specific skills in comparison to generic skills, and these considerations are not always comparable. See Pearce Report, supra note 42, vol l, 113–114, and National Board of Employment, Education and Training, supra note 10, at 36.

47 Any activity undertaken while awake can be described in terms of its propensity to introduce, or develop skills. For example, teaching by means of dictating well-worn lecture notes might be defended in terms of encouraging the development of written skills, and of professional skills including sitting quietly and respectfully while an authority figure drones on monotonously.

48 This approach makes certain assumptions about students’ career aspirations, employers’ expectations, and the proper venue for learning professional competencies and skills. All of these issues draw on long standing debates about the purposes and functions of institutional legal education. See J Thomson, Objectives of Legal Education — An Alternative Approach (1978) 52 Austl LJ 83, at 83, 88–90, J Wade, Legal Education in Australia — Anomie, Angst and Excellence (1989) 39 J Legal Educ 189, at 194–5; W Twining, Pericles and the Plumber (1967) 83 LQ Rev, 396. We will assume that it is relatively uncontroversial that at least one of the purposes is to introduce students to basic competencies required in legal practice.

49 Skills activities include simulated activities conducted in the formal institutional setting, from simple written activities such as letter writing and contract drafting, to more complex activities such as mooting and negotiation role plays, as well as activities undertaken in the “real” world, such as involvement in live client files at legal clinics.

50 Basing skills education on a description of what presently occurs in legal practice promotes and sustains an uncritical acceptance of the status quo. There remains some truth in Twining’s identification of the “relatively humble small-town solicitor” as the paradigm organising figure in this conception: supra note 47, at 399. Similar concerns were expressed in relation to the identification of authenticity in cognitive apprenticeship, see supra note 15.

51 Le Brun, & Johnstone state that “the assessment of student skills ... appears to be under-researched, under-theorised and, at times, difficult”: supra note 6, at 215; cf W Twining, Taking Skills Seriously (1986) 4 J Prof Legal Educ 1, at 1. We think this concern is less significant than developing learning objectives, and matching the learning objectives against learning activities in a directed way.

52 Teachers who make decisions concerning the skills which they want to develop in moots should advise students what skills are important to the exercise, and how these are related to the subject.

53 The first year moot concerns negligence law, the second year moot constitutional law. This is the third year moot.

54 FCT v Whitfords Beach Pty Ltd [1982] HCA 8; (1982) 150 CLR 355.

55 FCT v The Myer Emporium Ltd [1987] HCA 18; (1987) 163 CLR 199.

56 Hence, the first hearing in the formative moot experiment was called the Directions Hearing.

57 The rebuttal issue is discussed in the next section.

58 We do not recommend that the formative moot replace the traditional moot. Formative moots are logically employed after students experience the traditional moot.

59 For instance, the outline of argument, the directions hearing, the written submission, the case in reply and the main hearing.

60 Some of the problems considered the notoriously complex, but largely unlitigated deemed disposal provisions of Part DI.4 in S 160M(3)(a), (6) and (7).

61 Some teams were made up of less than three members, because of remainders. These teams were not required to deliver a rebuttal, although they could do so if they chose.

62 Six per cent of respondents thought that the allocation was too high; the remaining 15 per cent thought it too low.

63 Seven per cent thought it too long, 14 per cent thought it too short.

64 A few stoics in the early groups readily acknowledged this.

65 Especially in relation to clarification of facts.

66 One might consider this function of the directions hearing to resemble a process of peer feedback.

67 The reader should note that the population of students responding to these questions on rebuttal was smaller (n=32), consistent with the specialisation which we encouraged for dealing with rebuttal.

68 The variance of this period naturally was quite high. Much depended on how teams divided up arguments. Of course, watertight submissions also attracted fewer questions.

69 Eight per cent of respondents thought the questions were too difficult, 18 per cent thought the questions were somewhat unfair.

70 R Eggleston, Similar Facts and Bayes’ Theoran (1991) 31 Jurimetrics J Law, Sci 8 Tech 275; LM Froeb, & BH Kobayashi, Naive, Biased, yet Bayesian: Can Juries Interpret Selectively Produced Evidence? (1996) 12 JL, Econ, & Organization 257; P Bergman, & A Moore, Mistrial by Likelihood Ratio: Bayesian Analysis Meets the F-word (1991) 13 Cardozo L Rev 589.

71 D Baird, R Gertner, & R Picker, Game Theory and the Law (Cambridge: Harvard, 1995).