Ferguson, Anneka; Lee, Elizabeth --- "Desperately Seeking ... Relevant Assessment? A Case Study on the Potential for Using Online Simulated Group Based Learning to Create Sustainable Assessment Practices" [2012] LegEdRev 6; (2012) 22(1) Legal Education Review 121

DESPERATELY SEEKING ... RELEVANT ASSESSMENT? A CASE STUDY ON THE POTENTIAL FOR USING ONLINE SIMULATED GROUP BASED LEARNING TO CREATE SUSTAINABLE ASSESSMENT PRACTICES

ANNEKA FERGUSON^* AND ELIZABETH LEE^**

I INTRODUCTION

Over the last couple of decades in Australia a number of factors have set the scene for a radical rethinking of the way practical legal training should occur between the completion of an undergraduate degree and admission to legal practice.

Firstly, there was a shift from practical legal training on the job, based on articles of clerkship (‘Articles’), to practical legal training within a tertiary education setting. Secondly, the growth of the internet has led to the creation of a market imperative that online practical legal training is provided. Thirdly, the student population has developed socially and economically in such a way that it is now demanding flexible, but relevant, legal education that will accommodate full-time work, families and other social and community commitments.¹ Finally, the shift to institution-based rather than workplace-based practical legal training has led to the development of competencies as the basis for assessing whether students are ‘ready’ for practice. These factors potentially put many assessment regimes at odds with the leading educational theories on the role that assessment should play in student learning both within and beyond the institution.

Since January 2010 the Australian National University Legal Workshop’s response has been to provide practical legal training by placing students into ‘firms’ of ‘junior solicitors’ to carry out legal transactions in an online simulated legal environment.² Over a period of 18 weeks the students are required to complete and respond to all ‘feed-forward’ and feedback on their work until they reach a competent standard on all tasks across the compulsory APLEC³ competency content areas of Civil Litigation, Commercial and Corporate, Property Law, Ethics and Professional Responsibility, Trust and Office Accounting, and the practice management areas of Lawyers Skills, Problem Solving and Work Management and Business Skills.

As an adjunct to embracing this simulated legal environment and enforced group work, the purpose and identity of assessment has become less focused on examining a student in a summative sense to check for competence. It has, rather, taken on a drifting, less tangible (but no less significant) role that is more akin to the ‘formative-plus-student self-assessment’ model for assessment espoused by Boud and Falchikov.⁴

In this brave new simulated world, the role of assessors and assessment evolves (and sometimes seems to disappear altogether). Assessors now use so-called assessable tasks as a catalyst to interact with students and mentor them to a point where they can practically demonstrate professional behaviour, in context, in a way that encourages ongoing development and reflection along a continuum of learning rather than simply assessing them for an arbitrary competency on silos of content knowledge. Layering this assessment environment with the dynamics of a group⁵also means that peer assessment and individual reflective assessment necessarily distorts and even supplants the traditional summative/formative assessor’s role as ultimate determiner.

This article will examine whether this change to an online, simulated environment has been effective in responding to the challenges created by changes to the student population and legal practice training environment, as well as whether these changes have created sustainable assessment practices that will contribute to the development of students’ professional identity beyond academia.

II  THE UNDERLYING MOTIVATORS FOR CHANGE

It is not the purpose of this article to provide an in-depth examination of the many legal education factors that have combined in Australia, nor to assert that practical legal training should be provided differently. However a brief examination of the elements particularly noted in this article is warranted. This provides the context for the particular changes to the way in which practical legal training is delivered at ANU Legal Workshop.

A The Shift from Articles Based Learning to Tertiary Institution Based Practical Legal Training

In the 1970s, a new way of thinking emerged regarding the best way to provide practical legal training to the new legal profession. Most significantly, the Ormrod Report promoted a move away from what some people saw as an ‘antiquated’ apprenticeship system of Articles to a replacement with institutionalised practical legal training.⁶

The option of Articles is still available in Victoria,⁷ Western Australia,⁸ Queensland⁹ and the Northern Territory.¹⁰ However, there is an ever-increasing shift to the provision of practical legal training through educational institutions, due to employers wishing to outsource the assessment component of their practical legal training to educational institutions or to employers being far less likely to take on articled clerks when it is perceived that young lawyers will not stay with the firm for long enough to justify the investment of time, money and resources.

B The Growth of Online Learning

By the end of the twentieth century, and certainly by the beginning of the second decade of the twenty-first, the availability of the internet as a tool for providing learning online simply cannot be denied. Most Australian tertiary education institutions have recognised this, to some extent, by ensuring they have online information, enrolments and even learning management systems to promote, administer and complement traditional modes of teaching. External learning courses (or what used to be ‘learning by correspondence’) now utilise the internet to provide opportunities to students undertaking courses while off campus.

The extent to which universities have been innovative in their utilisation of the internet as a teaching resource in its own right has (and still does) vary considerably between and, even within, institutions. At the one end of the innovative online scale, the lecture/seminar/assessment mode of delivery has simply been approximated as closely as possible using the internet tools available: lectures are recorded and provided to students online (whether by live streaming or as a link to an audio file), and seminars occur through online conferencing or are supplanted by written work. At the other end of the scale, extensive research and development has occurred to provide online learning experiences that embrace new technologies in an attempt to provide a better educational model online than is often provided in the ‘face-to-face’ model. This includes digital gaming, simulations, multimedia presentations, and interactive assessment or learning tools.¹¹

Irrespective of an institution’s position on this operational scale, the recognition is that to remain viable in a business sense (at the very least), online learning as an option is unavoidable — even if debate rages as to its value.

C  Student Demands for Flexible, Relevant

Legal Education

The modern consumer-based economy and egalitarian approach to education means that students are also more evidently discerning and vocally demanding about the way in which their law degrees and practical legal training are delivered. Larger numbers of students are choosing to (or need to) work, mostly full-time, and often in legal offices, while completing their degrees. This earlier financial autonomy, plus exposure to the realities of the workplace, appears to highlight for many students the juxtaposition between their ideals of contributing to the social good, their learning in the academy and the requirements of being a successful employee.

For example, a snapshot of 338 students¹² surveyed at the beginning of their enrolment in the Professional Practice Core Practice Course¹³ at ANU’s Legal Workshop in June 2012 shows the following:

• 93.1% of participants are engaged in some kind of paid employment (with 61% being engaged in full-time work).

• When asked to select as many statements as were applicable to accurately reflect their experience of undergraduate law school:

− 74% chose ‘Encouraged analytical thinking’;

− 72% chose ‘Challenging’;

− 62% chose ‘Assessment-driven’;

− 59% chose ‘Interesting and thought provoking’; and

− 50% chose ‘Academic/theory focused’.

• Less than half of the students thought their undergraduate experience ‘encouraged/development ethical practice’ (37.3%).

• Very few (less than 15.4%) saw their undergraduate degree as ‘community/social welfare focused’. A disturbingly small minority suggesting that their experience ‘made them dislike the legal profession’ (10.4%) and ‘made the legal rules appear confusing’ (17%).

• When asked to order eight statements about the purposes of assessment, the top four statements chosen by the majority of participants were (in order):

− ‘Reaching a competent standard for legal practice’;

− ‘Understanding the content of the law’;

− ‘Regurgitating the information provided in the course’; and

− ‘Doing the work required in the assessment task’.

• Conversely, the respondents ranked the following statements as less applicable to their undergraduate assessment experience:

− ‘Providing creative or practical solutions to legal problems’;

− ‘Providing theoretical or abstract academic responses to legal problems’;

− ‘Demonstrating improvements in legal skills’; and

− ‘Reasons unknown to me — marks/grades seemed arbitrary and not connected to my learning’.

• Of the total participants, 50.9% had a strong/definite intention, and a further 27.2% of participants had a moderate intention, to enter a law-related career.

• Significant groups of students self-reported that they did not know anything, or were not abundantly confident that they knew anything, about being a legal professional at the end of their undergraduate law degree. For example, 31.4% of respondents on average felt that their unknowns would somewhat outweigh their knowns, and 36.1% strongly felt that most of what it would mean to be a lawyer would be unknown to them on admission. Only 31.4% felt that little will be unknown to them on admission.

In support of these findings, and as a further reflection of law student discontent, movements such as the ANU Law School Reform Committee have emerged. The Committee’s extensive report, Breaking the Frozen Sea: The case for reforming legal education at ANU,¹⁴ provides some insight into what this group of students (approximately 350) saw as the deficiencies in their undergraduate law degree. While the findings in this report have not been unanimously embraced by all students within the institution and is not necessarily accepted by the faculty, the robust methodology in its creation suggests that it is a legitimate source of information regarding law student views on the need for change in the manner, relevance and content of their legal education. For example, the opening pages of the report criticise the content and form of teaching and assessment. It suggests that the teaching and assessment mechanisms being used have led to disengagement and promotion of individual competitiveness rather than collegiality, and there is a lack of opportunity to engage with the social good. To quote:

We discovered that the ‘law’ was a series of rules, handed down by old men on the bench to lawyers who ‘neutrally’ applied it. Law school was a process of learning and memorising what ‘is’ – not dreaming of what could be, not arguing for what should be. Our lived experiences were irrelevant to our learning. Law hovered in a strange vacuum, outside of society, culture, politics, and even history. At the end of the semester, we would take exams, worth up to 100%, in which we would copy out these rules. A bell curve system mandated that more than half of us could receive no higher than a credit. I learned that my success depended upon other students receiving poor grades.

Even though I had wanted to use my law training to change the world, most days I would have preferred to stay in bed. I could not engage with the two-hour lectures on dry, doctrinal subject areas, in which I passively copied down pages of legal rules. With this disengagement came feelings of guilt and self-doubt. There was also the sense that my participation made no difference anyway. Sometimes I put in extra effort and got credits, other times I crammed an entire course at the end and got distinctions. By third year, I took a full time job. I did not attend many of my courses and I stopped buying the textbooks. Using the condensed summaries that did the rounds each semester, I taught myself several courses from beginning to end three days before the exam. None of this changed my marks ...

Instead of learning to collaborate with others — that it is amazing what can get done when it doesn’t matter who gets the credit — we learn that we should compete, fiercely self-promote and reproduce hierarchy ...¹⁵

This sentiment does not seem to be confined to these particularly disenchanted students. It is also evident in the work that colleagues Kath Hall, Molly O’Brien and Stephen Tang¹⁶ (as one example)¹⁷ have been conducting on student wellbeing and the detrimental impact that the first year of law school (as studied) appears to be having on law students’ mental health. For example, it may be that some aspects of law school, as opposed to other disciplines, are encouraging an adversarial, dispassionate way of thinking that leads to a ‘mismatch between students’ anticipated experience of law school and their lived law school experience’.¹⁸ Furthermore, a pilot study of newly admitted lawyers in the Australian Capital Territory has found that this experience does not match the collegiality and the wider conception of professionalism that newly admitted lawyers see as fundamental to their success in early practice.¹⁹ Where empathy and resilience should be fundamental to the development of a professional personality, competitive individualism and detached, legally rational thinking are being promoted instead.²⁰

In short, it is possible that the highly competitive, individualistic, rational form of law that is often taught in law schools is not only disenchanting the student population, but is also having detrimental effects on its wellbeing and preparedness to meet the needs of the profession when entering it.

On a broader scale, the Graduate Course Experience Survey Report from 2010,²¹ which records experiences of graduates from a number of national universities, indicates: (1) students think their law degree workloads are inappropriate; (2) there is insufficient teaching staff feedback and support; and (3) only 56.5% of respondents thought the assessment they completed as part of their law degree was appropriate.

In summary, there is a general feeling by the law student population that, to meet its needs, legal education and assessment in legal education needs to be responsive and relevant to a diverse legal profession. And, broadly speaking, current methods of teaching are not meeting this need.

D  Competency Ticking versus Assessment

as Learning

In addition to the factors mentioned above, a growing body of work has examined the impact of assessment on students’ behaviour, wellbeing and learning outcomes. The traditional summative assessment regime of 100% closed-book exams, while often justified as providing a necessary warranting function, has nonetheless been roundly criticised by numerous writers and educators for not providing the skills for lifelong learning²² and contributing to fear of assessment²³ and lack of preparedness for the realities of legal practice.²⁴

Whereas some institutions have focused on developing learning based curriculums in order to address issues of disengagement and stunted learning outcomes within their student populations, this has often been done without also recognising that the assessment regimes within a curriculum are a significant motivating factor in the nature of student engagement. Driessen and Van Der Vleuten, for example, eloquently summed up the relationship between curriculum learning objectives and assessment as follows:

As the students’ academic success is defined by the examination program, this is at the top of their agenda. To students, the examination program is the actual curriculum. No problem exists if the educational objectives as defined in the curriculum are matched by the objectives of the assessment program. However, in the event of a mismatch, the student assessment system will prevail over the actual learning, irrespective of the loftiness of the curriculum objectives.²⁵

Thus it seems that in order to address this motivation, it is necessary to incorporate assessment effectively as a learning opportunity within the curriculum or to create an assessment regime that does not define academic success by performance in examinations only.

This is not a new concept in educational practice and many experienced writers have looked at assessment that is formative rather than simply summative in function.²⁶ Furthermore, writers such as Boud have taken the discussion of formative learning and assessment and extended its relevance beyond academia to espouse the need for an assessment regime that functions as the catalyst for students to become active players in assessment rather than passive receivers of marks or feedback so that they can ultimately develop the lifelong self-assessment skills to make complex judgements about their own work.²⁷ In short, assessment needs to have sustainable outcomes relevant to the development of a legal professional, and to the legal profession.

However, irrespective of the dubious value of assessment to simply being able to warrant students as having reached competencies, the reality of the Australian situation is that all practical legal training providers do need to ensure they are able to certify that students who have completed their courses have met the requirements of the APLEC competencies.²⁸ While this can be seen as a constraint on innovative assessment practices, the case study in this article suggests that these requirements can still be met (and possibly, more adequately) while providing students with relevant, practical legal training.

III  ADDRESSING THE CHANGES: HOW HAS LEGAL WORKSHOP RESPONDED?

By the end of the first decade of the twenty-first century, ANU Legal Workshop had already started responding to some of the changes to the nature of the student population and legal practical training environment. For example, Workshop had already recognised the business need to be online in order to provide flexible learning opportunities for postgraduate law students by moving to become an online provider of practical legal training. However, much of this change involved taking the face-to-face silo modules of compulsory content areas and replicating the teaching methodologies and assessment in an online environment. While, in many respects, this model was functional, the factors outlined above were braying for a more assertive change in direction in order to more appropriately meet the needs of students and the profession for real authentic, integrated, professional and flexible practical legal education. In short, a more innovative response was required.

The evolution of a significant change in educational direction by any institution is never simple. As the purpose of this article is not to do justice to or describe the mechanics, trial and trauma of this change in depth, it is sufficient to indicate that the work of Boud and Falchikov sparked an interest in changing the strategic direction at ANU Legal Workshop to achieve sustainable assessment and lifelong learning. Once this occurred, a search began for an educational framework to meet this need within the constraints of a competency and institutional based online environment. A further encounter with Paul Maharg and the SIMPLE project team at Strathclyde University²⁹ provided a new pedagogical direction that ticked many of the required boxes needed to create sustainable assessment. SIMPLE was described as a:

SIMulated Professional Learning Environment [that] enables students to engage in online simulations of professional practice. Its special pedagogy is based on transactional learning: active learning through performance in authentic transactions involving reflection in & on learning, deep collaborative learning, and holistic or process learning, with relevant professional assessment that includes ethical standards.³⁰

Some of the other mechanisms enabled by SIMPLE were for students to receive feed-forward and feedback on work in an ongoing fashion and the opportunity to redo work until such time as it had reached a competent standard. In addition, the SIMPLE environment enables students to work in groups to complete tasks; exposes them to legal content areas in context; and presents both content areas and assessment in an integrated fashion rather than as silo subject areas or assessment timetables.

The result of this fusion of Boud/Falchikov and SIMPLE was the development of an 18-week, entirely online, professional practice core course. This course, named the Professional Practice Core (PPC) Course, integrated the previously individually taught areas of property, commercial litigation, civil litigation, trust accounting and ethics³¹ as well as the barely taught practice management competencies into a single course environment. This integrated learning environment (ILE) engages students simultaneously on two levels:

● An ‘in role’ simulation environment comprised of a virtual office space (VOS) where students work in groups with up to four other students to create a team of lawyers engaging ‘in role’ in transactions in a virtual firm environment. In this space students work to complete transactions involved in conducting files with direction and support in the form of task allocation, and receive feed-forward and feedback from Senior Partners, Associates, clients and other characters that are representative of a lawyer’s experience in practice (‘in role’ learning).³²

● An ‘out of role’ support web-site using the university’s learning management system allows students to engage with each other and teaching staff to access and process additional resources and information to support their ‘in role’ learning. In this area students are also required to complete individual assessment tasks (‘out of role’ learning).

Wherever possible, students are encouraged to strive to work and resolve problems ‘in role’, as it is expected that through this process they will be able to develop and demonstrate their problem solving skills in the context of legal practice.

Assessment within this environment is integrated within the tasks, so that it is occurring in an ongoing manner and involves substantial formative feedback and feed-forward. Students must operate as a group to discern the tasks, problems and issues that need to be addressed and then provide and resubmit their responses to their supervisors until such time as they reach a competent standard. The expectation level as described to students is that:

When you enter practice, one of the only ways you can know whether you have met the expectations of your employer/colleagues is whether they indicate that your work is of a satisfactory standard to be sent out under your employer/firm’s name. You will often find that you are required to re-draft documents a number of times before they reach the standard expected of the law firm. The virtual office space within the Professional Practice Core operates in the same way ...

... The focus of the ‘in role’ firm is to maximise your learning and to enable the achievement of competency. To achieve this, your firm will receive both ‘feed forward’ and ‘feedback’.

As you and your firm are still developing your skills for practice, it may be that you will have to re-submit your work to a Senior Partner/Associate or Accounts Personnel on several occasions before it is deemed competent. This is a normal part of developing and enhancing your professional skills in the virtual office environment. Your ability to respond to and embrace the feedback/feed forward that you receive in a constructive fashion is an indication of your professionalism.

If the work provided to the Senior Partner is of a competent or higher level performance standard, the Senior Partner will approve the work for sending to the client and/or the other firm acting in the matter. At this point your firm will know that you have reached an expectation of at least competent in this item of work.

Once it is indicated by the Senior Partner that you have competently completed all tasks in their specific area of practice, all members of your firm will receive a competent rating ... If your firm consistently achieves a higher standard of work during their work in an area of practice, this will be recognised as a Higher Level Performance (HLP) rating ...

Firms must achieve an overall level of ‘Competent’ or ‘Higher Level Performance’, within each area of practice, in order to complete and pass the PPC.³³

In addition to the Virtual Office Space used for the ‘in role’ work, an ‘out of role’ learning website is also provided. This site is a backup to student learning and offers: a repository of helpful links and resources; an ‘out of role’ Q & A and discussion forum; and information about and the location for all individual ‘out of role’ assessment tasks.³⁴

‘Out of role’ assessment takes the form of formative quizzes, reflective writing pieces, individual transactional tasks (such as conducting an interlocutory application by web conferencing or the completion of a set of legal practice trust accounting books). Importantly, ‘out of role’ assessment is not seen as more worthy, or as an alternative to ‘in role’ assessment. Rather, it is designed to complement the work being done in the ‘in role’ virtual office space (VOS) of the student firm group.

The design of the ‘out of role’ assessment is aimed at motivating individual learning and achievement, and also rewarding those who have actively engaged in the group work. For example, if a student has not engaged with the group work, they will find it far more difficult to complete their individual tasks. Thus, while the individual assessments are designed to enhance and complement the group learning experience, there is an individual warranting function built in as well — we can warrant that individual students have achieved a competent rating against the APLEC admitting requirements without ‘riding on the coattails’ of their peers in the group work.

In addition, assessment is only graded as being: Not Yet Competent (NYC), Competent (C) or Higher Level Performance (HLP) and does not involve numerical marks or any form of bell curving. Students must reach at least a competent standard on all group and individual work in order to successfully complete the course; and neither group nor individual work is rated higher than the other.

The end result of the combined regime of ‘in role’ and ‘out of role’ assessment tasks is that students are exposed to a plethora of ‘assessment’ formats in an ongoing fashion throughout the course. This allows assessment to play a number of roles both in conjunction with and completely integrated into the curriculum — so that at times (for example, with many of the out of role assessment tasks) the assessment is very obviously badged as an assessable task and yet, at other times, assessment is so integrated within the required tasks that students cannot help but be assessed by simply engaging with the task.

This mixture of assessment styles allows assessment to appear as both summative (by allowing us to externally warrant that students have demonstrated at least a basis competency) and formative (the assessment is so integrated within the required tasks that it is central to the learning process). However, it is also arguable that integrated simulated assessment, as it appears within the PPC Course, is one of the few models of a course assessment regime within an institution (rather than, say, an apprenticeship) that provides the opportunity for students (and very nearly legal professionals) to develop sustainable self-assessment practices. So is the assessment regime within the PPC Course the ultimate combination of SIMPLE and Boud?

A SIMPLE-y Boud?

In promoting sustainable assessment, Boud and Falchikov have noted that the traditional

discourse of assessment draws strongly on the metaphors of acquisition and judgement. It sits less easily with the metaphor of participation that is being increasingly used to characterise workplace learning.³⁵

At ANU Legal Workshop, the adoption of the SIMPLE based model for the Professional Practice Core Course was essentially aimed at bringing back a more practical, participation based learning system. But, the question remains: did the adoption of the Integrated Learning Environment based on Maharg et al’s SIMPLE create a sustainable assessment regime?

Boud and Falchikov have suggested that a sustainable assessment regime could include a combination of the following characteristics:

1 Engages with standards and criteria and problem analysis;

2 Emphasises importance of context;

3 Involves working in association with others and authentic representations and productions and promotes seeking appropriate feedback;

4 Fosters reflexivity and considers risk and confidence of judgement;

5 Builds learner agency and constructs active learners; and

6 Requires portrayal of outcomes for different purposes.³⁶

Each of the six characteristics is considered in turn below. By analysing the case study of the PPC course and assessment structure against each of these characteristics, we may gain some guidance as to whether a sustainable assessment regime has been/is being created.

B Engages with Standards and Criteria and Problem Analysis

Within the ‘in role’ learning environment students do not receive tasks according to a timetable with set criteria next to them. Instead they receive letters of instructions from clients or videos of a client interview or an internal memorandum with accompanying authentic documents from their Senior Partner. These outline an issue or give a short description of the legal task required. This means students are engaged in identifying, from the information presented to them, the parameters, structure, form and deadlines for the tasks received. They then need to do the appropriate research individually and as a group to scaffold their knowledge in order to complete the tasks. Options for appropriate assistance in scaffolding their knowledge are provided through:

• their practice mentor (a legal practitioner assigned to each firm to assist with mentoring the group dynamics and practice management and professionalism skills);

• the ‘in role’ characters such as associates, clerks and senior partners (‘played’ by a legal practitioner assigned to the firm) and fellow group members;

• the ‘out of role’ discussion forums and resource library; and

• external sources of information such as the internet, previous undergraduate course notes and texts and (if they work in a legal environment) their own work colleagues.

Students must evaluate the best avenue for seeking assistance within the resources available. For example, it is not uncommon for questions directed to the ‘out of role’ discussion forums in the student/teacher space to be answered along the lines of ‘It would be more appropriate to ask your Associate in the virtual office space this question’ or, more fundamentally ‘Have you tried using x search function in the resource library? There are substantial resources available there’; as opposed to ‘Here is the direct answer to your question (subtext: you don’t need to think about it anymore yourself)’.

Furthermore, within the group and over the course of their interactions with clients and Senior Partners/Associates, the students need to develop the skill of identifying the criteria and standards for a competent piece of work. Broad criteria are available to students in the ‘out of role’ students’ space regarding the nature of appropriate professional writing styles and analysis. However, students must engage with the work provided in good faith and thus learn the standards to be expected by the responses they receive in the feedback and feed-forward.³⁷ The standard that is expected is that of an entry level lawyer — they are not expected to be experts — but they need to demonstrate that they have engaged with the basic skills required to operate at a supervised level on day one in practice.

In addition, because the simulated tasks are completed as a group, the group of students themselves will also self- and peer-assess each other in a manner that sets the standards for the group. For example, it is not uncommon to find groups who define themselves either by the fact that they expect to achieve a higher level performance in all work or a merely competent in all work; and to adjust their responses and expectations of each other accordingly. On the face of it, it could be concerning that a group will decide to be ‘merely competent’. However, because this standard is actually quite high — a considerable amount of identifying issues, processing solutions and knowledge is required to achieve a competent rating across the board — it is not the immediate problem it may first appear. In fact, we have observed that an interesting self-assessment occurs with students who decide just to be ‘merely competent’. For example, we have observed students who have said they are choosing to prioritise work or family commitments over their study, and accept that gaining a competent rating is sufficient for them to achieve the skills they need. Similarly they may strategically decide that a higher level work may not always be required in an environment where achieving an outcome efficiently and prosaically is all that is required.

C Emphasises Importance of Context and Promotes Seeking Appropriate Feedback

In a practical legal training environment, context is fundamental. Students need to learn to join together their knowledge from undergraduate studies with the relevant skills required to make a competent and professional lawyer, such as effectively communicating, researching, advising, negotiating, and advocating. By requiring students to work in groups within a simulated law firm environment to complete entire matter files, they must learn to locate issues in the context of the ‘messy’ legal file and the functioning legal business. While we do not simulate the ‘nasty’ work colleague in this environment (on the basis that professionalism should be modelled by adherence rather than exception), students still have to ‘pick through’ the complexity of a roughly taken file note (for example) to identify the legal issues. Furthermore, in order to complete tasks, students must also identify the further information required; negotiate the context of a busy legal practice by only seeking an appropriate level of assistance from others; and recognise (and become comfortable with) the fact that there may not be simply one lecturer-defined, predetermined solution to the problem as presented.

In addition, the group of students must assess among themselves what aspects of their work require feedback from each other or from the Associate or Senior Partner. Ultimately, a confident team of students may reach a point near the end of the course where they do not race to meet the optional deadline of providing work to the Associate for feed-forward because they feel that their work is of a standard to go directly to the Senior Partner for sign off.

Furthermore, as a significant number of the ‘out of role’ individual assessments are extensions of the simulated legal context the students are experiencing in the ‘in role’ group work, even the individual assessments are not removed from the ‘realities’ or legal practice. For example, in the property practice area, the individual assessment is completed by the student providing an oral advice to the ‘client’ on the conveyance process their group or firm is completing in the virtual office space. Or the civil litigation practice area may require students to individually complete an interlocutory application before a ‘registrar’ on the basis of the statement of claim/defence they had prepared as a group in the virtual office space.

Even the individual assessments that may not directly relate to the specific matters in the virtual office space are substantively based on the students’ experience of that context. For example, the trust accounting bookkeeping assessment was frequently viewed as irrelevant to legal practice when the course was taught as a subject on its own. However, it is rare to hear this complaint now that students have had to engage in trust accounting issues that emerge in their various substantive legal practice areas, such as gaining access to money for fees or legal disbursements in trust for the work they are completing.

D Involves Working in Association with Others and Authentic Representations and Productions and Promotes Seeking Appropriate Feedback

The PPC Course as described thus far emphasises the importance of students working in association with others. As a starting point, the students are required to work with the same student group of four for the group work and assessment within the ‘in role’ Virtual Office Space for the entire 18 weeks of the course. As the group is required to submit only one combined response to any task within the virtual office environment, they must necessarily work together to combine resources, draft, edit and re-edit each other’s work in order to complete the matter file. The need to work together on such items is reinforced by the students having to: accurately record their participation (in a format that is viewable by their group members); be ‘on top of’ the group responses so that they can participate fully in the file; and understand the group work so that they can successfully complete individual assessment. As a result, the egalitarian transference of knowledge and assisting each other to discuss solutions to issues is actively encouraged. Importantly, this environment requires them to demonstrate and develop the essential skills of how to give and receive feedback on all work from their immediate peers in an ongoing fashion.

Based on the observations we have made of student group participation, the impact of having to submit to other students to review cannot be underestimated in terms of motivating students to engage and to develop self-assessment skills. Often students have been trained at undergraduate level to validate themselves using marks received from a distant arbiter. However, once they have to present their work to peers within the PPC Course, they frequently spend considerable time and effort evaluating their own work before handing it up for review by their student group members. Then, through the responses they receive from their group members, they are able to develop the ability to validate both their work and their self-assessment capabilities.

Student groups are also ‘paired off against’ each other on either side of the matter files they are dealing with. As such, the student groups must negotiate how to conduct a matter when there are ‘real’ solicitors acting for the other side and, as such, they must deal with situations where the solicitors may not act as they had hoped — not returning work on time, not advising their clients as expected, or not seeing the issues as they see them. This has led to some interesting results which, at times, have highlighted the fundamental difference between providing an advice on a scenario out of context and how one should behave when one is dealing with ‘real people’ on the other side. For example, in one instance, Firm A wrote to their opposing Firm B, along the lines of: ‘We have advised our client that you will get your client to proceed with the action. Accordingly, you need to make sure your client proceeds with the action so it is consistent with what we have told our client is going to happen’ (and they didn’t see what was wrong with that)!

In addition to working with other students, the student group and individuals must work in association with the other third party ‘characters’ within the virtual office environment — clients, clerks, Associates, banks, Senior Partners, administrative staff — to ascertain and complete the tasks required to complete the matter file to a competent standard. These ‘characters’ are ‘played’ by real, often currently practising legal practitioners, who are casually employed and trained to play the roles. Accordingly, the student groups must ensure that they are working with these characters in a professional and appropriate fashion: ensuring that all communications are courteous and that they do not overstep the useful/relevant question boundary that would be appropriate to asking these kinds of people questions in a legal practice context.

Furthermore, the students can engage with lecturers and employed legal practitioners within the ‘out of role’ space to further their understanding of concepts and the practice of law. Most significantly, the students engage with their practice mentors (legal practitioners employed to mentor the firm groups) to develop an understanding of the legal practice context and what constitutes fundamental elements of practice management.

Finally, because the ‘in role’ problems or matters and tasks are presented to students in a manner that simulates the allocation of work in legal practice, students are given the opportunity to develop and gain a more advanced understanding of how the often very theoretical knowledge they obtained in their undergraduate degrees may actually be presented to them in practice and ‘real life’ situations.

As such, the experience of students within the ‘in role’ virtual office environment (in particular) is aimed at being an authentic representation of the kinds of activity a legal professional may undertake at an entry level. All documents are constructed to provide as accurate a representation of the documents encountered in practice as possible — internal memos, forms, office manuals, title searches, cheques, court documents and evidence are carefully produced to ensure that authenticity represent those encountered in practice. In addition, names are carefully chosen to ensure they are authentic and represent the diversity of community that lawyers may encounter in practice (gender, ethnic background, interests outside the firm), and Senior Partners are encouraged to present themselves as people with sometimes variable habits that need to be discerned and accommodated — for example, only available at certain times, ‘busy’ with a ‘trial’ and the like.³⁸ While this need to discern different requirements between the Senior Partners they encounter in the course certainly contributes to the complexity of students’ engagement, it is arguably appropriate to provide this experience to students in this safe environment, as it is a necessary requirement of practice to be able to navigate such different personalities.

E Fosters Reflexivity and Considers Risk and Confidence of Judgement

Students undertaking the GDLP and the PPC Course have completed (or are in the final semester of) their undergraduate law degree. As such, they have completed at least three, but more likely five, years of legal content, very often alongside another degree.

The integrated, authentic PPC simulations require students to make linkages between:

• the content knowledge obtained in their undergraduate degrees and private work experience;

• the resources available within the ‘in role’ and ‘out of role’ spaces; and

• their independent research and the knowledge and experience of the people they are working with, to construct solutions to the problems they have identified both within the group ‘in role’ space and when completing their individual assessment tasks.

The problems provided in the virtual office context, by virtue of being authentic, often contain a significant amount of ‘messy’ information which needs to be interpreted, prioritised and evaluated before ‘answers’ can be provided. Similarly, as mentioned above, the transactions are not set up to have only one ‘right’ answer, but rather to encourage students to canvass the options that are available and ‘test’ one of these options within the safe learning space in order to ascertain how it may pan out in practice. Furthermore, they are told to ‘think outside the square’ and try creative responses and not be afraid that they will ‘fail’ if the response does not work because the assessment of competency is based on their skills and ability to reflect on why their response did or did not work rather than when they reached a certain pre-determined outcome.³⁹

For example, within one of the commercial transactions, the students had to negotiate a confusing web of interconnected directors and trustees duties to negotiate settlement of a conflict that had arisen between these duties in a manner that suited all parties (who wished to remain friends and business partners). Some students took a very litigious approach to the situation but then realised in the end of transaction debrief with their mentor that, by doing so, they had failed to actually meet their client’s goal of continuing to work together. As long as the students who took the litigious approach were able to recognise that other pathways may have been more successful in providing a solution to the client’s needs, they could still achieve competency for this area of practice.

As such, the simulated transactions encourage students to test their ‘old’ theoretical knowledge in a practical environment and then self-assess whether they have been successful in doing so within an authentic legal practice context.

The individual assessment regime complements this self-assessment process by providing reasonably quick and direct feedback to students regarding their individual understanding and progress and participation within the ‘in role’ work. For example, we have often observed that those students who come to our attention as struggling within the group work, either through the need for further skill development or in terms of participation, are often the students who do not meet the competency standards in their individual work. Conversely, a high-trust, highly functioning learning group of students within the virtual office environment will, more often than not, have this success reflected in their individual results for the individual assessments. This direct correlation between achievement in the group work and individual assessment achievement reinforces student confidence in the self/peer evaluations they necessarily have to make in their group work: they are able to validate the self-monitoring skills they are developing in the simulated environment by finding they are relevant beyond the confines of the virtual office environment.

F Builds Learner Agency and Constructs Active Learners

While it is difficult to suggest that the PPC Course actually allows students to create their own assessment tasks in order to build learning agency,⁴⁰ it is arguable that a number of features of the PPC Course assist in building learning agency and constructing active learners.

Firstly, the need for students to discern the required tasks from a matter file rather than simply doing the tasks outlined on an assessment timetable means that they do effectively have to create an assessable task out of the material provided and then construct solutions based on the resources and knowledge available to them.

Secondly, the fact that students have to continue to do work by responding to the feed-forward and feedback that they receive from fellow students and their Associate until they get it right is also motivation to be fully engaged with the work in the first place. For example, as there is no ability to just choose the best pieces of assessment to be included, or only engage with ‘the most significant’, or easiest pieces of assessment in order to ‘merely pass’, students are motivated to be engaged with the task from the beginning.

This motivation is further encouraged through the advertisement of the clear link between participation within the group work and the individual assessment, and by the fact that there is also a very evident link between the work required of them and the work they will need to complete upon entering the profession. The purpose of the tasks/assessment is not to reproduce knowledge that is only relevant to academia, but to produce documents from the information provided that are relevant to legal practice.

The competency based grading system also shifts the focus of students from whether they have received good marks or beaten their classmates to what they need to improve in order to meet the standard of work that is adequate to go out under the firm name.

G Requires Portrayal of Outcomes for Different Purposes

With the focus of the PPC Course on providing opportunities to develop exactly those skills required to complete the tasks of a legal practitioner, it is definitely arguable that the course ‘identifiably leaves[s] students better equipped to complete future tasks’.⁴¹ Indeed, students themselves have commented in the end of course evaluations that they feel better equipped to deal with going into practice now that they have had the opportunity to practise skills that ostensibly appear to be applicable to the profession they are seeking to enter.

Progress has yet to be made in providing students a platform for portraying these achievements to others in the form of portfolio. There is always room for improvement.

IV CONCLUSION: IS THE PROFESSIONAL PRACTICE CORE INTEGRATED LEARNING ENVIRONMENT WORKING?

The analysis of the PPC Course in relation to Boud and Falchikov’s criteria for a sustainable assessment environment suggests that, in this regard, the course has made substantial progress in providing a relevant, authentic, group-based learning environment that promotes sustainable assessment practices.

Beyond this, it would appear from the last two student and staff evaluations and the observations made by the convenors in the course (of which there are a number) that, on the whole, the PPC Course is providing an appropriately integrated course assessment regime to ensure that students are receiving relevant, practical legal training to meet the needs of the profession. For example, the two most recent student and staff evaluations of the course reveal the following:

• A majority (70%) of staff and students found the overall course to be sound (50%) or excellent (20%).⁴²

• A majority of the staff and students ‘found the simulations and practical activities valuable in their learning of practice’ and valued ‘the incentive to develop an understanding and activities of professional practice’.⁴³

• ‘Students identified the practical nature of the activities and artefacts of the PPC Course as representing its major area that contributed to the improvement in their legal skills. In the various areas of the PPC Course, the relevance of tasks that were providing the opportunity to engage in activities that replicated practice environments was valued’.⁴⁴

• A ‘significant number of students positively reflected on the benefit of undertaking a range of practical writing and drafting tasks. Many students observed such tasks provided a highly useful and relevant precursor to this form of work in practice environments. Others also drew on the challenges and learning of writing within and to other groups as it provided the opportunity to more rigorously assess the quality of individual writing and drafting tasks’.⁴⁵

• Previously raised issues of consistency of program epistemology are no longer appearing in the evaluation reports and appear to have been resolved.⁴⁶

• A majority of students found the simulation of the matters within the virtual office to be useful (45%) or very useful (21%).⁴⁷

• A majority of students were satisfied with the support they received from Associates, Senior Partners, the ‘out of role’ discussion forums and the resource library.⁴⁸

• A majority of students found the individual assessments to be useful (56%) or very useful (24%).⁴⁹

Notably, earlier issues that appeared in the evaluations of the first iterations of the course have stopped appearing in the more recent evaluations.⁵⁰ For example, there is no longer significant complaint about:

• students having dysfunctional groups affecting their learning experience;

• timeliness and type of feedback and feed-forward;

• student over-workload;

• perceived lack of quality of instructions; and

• design of the ‘out of role’ support website.

That these comments have largely disappeared from the evaluation results is an indication that the program has matured and that the consistency of staff training, standard setting and presentation of epistemology has improved.⁵¹

Having said this, the PPC Course is not perfect. Technology and software failures at critical times (for example, web conferencing not working properly in oral assessments) still cause considerable issues. And some students question the value of the reflective logs and debriefing processes as tools of learning. However, it is gratifying to see that many of the student criticisms of these tools are based on the fact that there is a dissonance between the authenticity provided by the rest of the learning environment and these tools: by reverse engineering this response, students are questioning their usefulness for learning, because they feel that the other tools being provided (simulations, individual assessments, feedback and feed forward) are more useful to their ongoing learning and readiness for practice.

Outside this survey response, and as if to contradict the above comments regarding a consistency of epistemology being achieved, an ongoing tension occurs whenever changes to the content or assessable actions within the course are canvassed, or when new staff are employed who are unfamiliar with the underlying premises of assessment within the course. This tension is between the need to warrant competency and the traditional style of validating teaching outcomes through assessment, and the ideal of assessing students in an integrated, sustainable fashion.

It seems that it is not unusual to have this kind of tension when implementing assessment strategies alongside innovative teaching models. As Driessen and Van Der Vleuten noted when they described the tension between the assessment regime and the problem-based learning approach that had been adopted at their university, ‘as usual, the assessment program gained the upper hand and slowly, but progressively undermined the problem based learning approach’.⁵² Within the environment of the PPC Course, this tension occurs between convenors’ desires, on the one hand, to be sure they are able to demonstrate that they have taught and certified each of the competencies and that students are ‘comfortable’ with the teaching support they have been given, and, on the other hand, the underlying pedagogy of an authentic simulated transactional learning environment which necessitates that student driven learning may be ‘messy’, not always comfortable and not always easily measured. As such, all changes that convenors wish to make to their assessment regime need to be carefully assessed to ensure that ‘the baby is not being thrown out with the bathwater’.

Furthermore, the sheer size of the course and the amount of staffing required to ensure responsive ongoing feedback and feed-forward to students both in and out of role is enormous. However, as many of these staff are employed from the legal profession on a casual basis, and the intakes of students choosing to participate in the program are now reaching in excess of 500 students each iteration, the staffing model is largely sustainable. In addition, by employing practising legal professionals to participate in the program, we are directly engaging the profession in the training of their junior colleagues in a meaningful fashion, thus reflecting back into this profession the sustainable models for mentoring and assessing that are being used in the course.

As convenors within the course, we have observed that the value of the integrated assessment regime within a largely simulated environment has been:

• student accomplishment through intrinsic motivation to learn, which is not determined by numerical grades based on decontextualised pieces of assessment;

• the ability to more actively and more accurately determine and encourage the development of important competencies, such as professionalism, that were not adequately addressed before;

• the ability to encourage students to examine their professional selves and their peers and thus become ‘self-assessors’ of their own ongoing professional development needs; and

• the ability to provide students with relevant practical legal experience where they learn from their mistakes whilst their mistakes can still be made in a safe environment.

In summary, assessment within the PPC is beginning to work in harmony with the innovations in curriculum development and delivery. At times, assessment may be hidden altogether; and the assessors, instead of being arbitrary markers, become mentors and role models for the legal profession. As such, they are able to use assessment to assist in creating competence for the profession whilst also completing the necessary warranting function required for entry level lawyers. It may just be an example of sustainable assessment in action.

^[*] Lecturer, The Australian National University, Canberra.

^[**] Lecturer, The Australian National University, Canberra.

^[1] ANU Law School Reform Committee, Breaking the Frozen Sea: The case for reforming legal education at ANU (2010) <http://lawschoolreform.com/files/lsr_breakingthefrozensea.pdf> .

^[2] This method of delivery has been influenced by the work of Paul Maharg and the Strathclyde team that worked on the SIMPLE project. The SIMPLE project is discussed in Part III below.

^[3] Australian Professional Legal Education Council, Competency Standards for Entry Level Lawyers <http://www.aplec.asn.au/Pdf/Competency_Standards_for_Entry_Level_Lawyers.pdf> .

^[4] David Boud and Nancy Falchikov, ‘Aligning Assessment with long-term learning, Assessment and Evaluation’ (2006) 31 Higher Education 399, 402 <http://www.numyspace.co.uk/~unn_evdw3/skills/current/papers/assess1.pdf>

^[5] K Barton and F Weswood, ‘From student to trainee practitioner: A study of team working as a learning experience’ (2006) 3 Web Journal of Current Legal Issues, Special Issue on Legal Education <http://webjcli.ncl.ac.uk/2006/issue3/barton-westwood3.html> .

^[6] Report of the Committee on Legal Education, CMND No 4595 (1971) as commentated on by Philip A Thomas and Geoff M Mungham, ‘English Legal Education: A Commentary on the Ormrod Report’ (1972) 7 Valparaiso University Law Review 87.

^[7] Law Institute of Victoria, Practical Experience Requirements <http://www.liv.asn.au/Practising-in-Victoria/Professional-Standards/Steps-to-Qualified-Practice/Practical-experience-requirements.aspx> .

^[8] Legal Practice Board of Western Australia, An Overview of Admission Requirements <http://admittance.lpbwa.org.au/An-Overview-of-Admission-Requirements> .

^[9] Queensland Law Society, Required study and training <http://www.qls.com.au/For_the_profession/Your_legal_career/Studying_law/Required_study_training> .

^[10] Supreme Court of the Northern Territory, Legal Profession Admission Guidelines, <http://www.supremecourt.nt.gov.au/lawyers/documents/Legal_Profession_Admission_Guidelines.pdf> .

^[11] See, eg, Sara de Freitas and Paul Maharg (eds), Digital Games and Learning (Continuum International Publishing Group, 2011).

^[12] This was 79% of the students enrolled.

^[13] The Professional Practice Core (PPC) Course is a compulsory, 18-week online course provided by ANU Legal Workshop which takes students through the core subjects required by the APLEC competencies: Civil Litigation, Commercial, Property, Trust Accounting, Legal Ethics and Practice Management. The PPC Course is discussed below.

^[14] ANU Law School Reform Committee, above n 1.

^[15] ANU Law School Reform Committee, above n 1.

^[16] See, eg, Molly Townes O’Brien et al, ‘No time to lose: Negative impact on student wellbeing may begin in year one’ (2011) 2 International Journal of the First Year in Higher Education 49; Molly Townes O’Brien et al, ‘Developing a Professional Identity in Law School: A View from Australia’ (2010) 4 Phoenix Law Review 20; and Kath Hall, ‘Do We Really Want to Know? Recognizing the Importance of Student Psychological Well-Being in Australian Law Schools’ (2009) QUT Law and Justice Journal (November 4); ANU College of Law Research Paper No. 09-31 <http://ssrn.com/abstract=1500155> .

^[17] Other examples from other jurisdictions include, but are definitely not limited to: in the United States, Judith Wegner, ‘A Legal Education Prospectus: Law Schools & Emerging Frontiers: Reframing Legal Education’s “Wicked Problems”’ (2009) 61 Rutgers Law Review 867; Karen Sloan, ‘Reality’s knocking: The recession is forcing schools to bow to reality’ (2009) <http://bestpracticeslegaled.albanylawblogs.org/2009/09/07/national-law-journal-reports-on-carnegie-and-bp-reforms-at-duke-ucla-dayton-and-others-egje/> Roy Stuckey et al, Best Practices for Legal Education (Clinical Legal Education Association, 2007); William M. Sullivan et al, A Vision and Road Map: Educating Lawyers — Preparation for the Profession of Law (Jossey-Bass, 2007) (‘The Carnegie Report’); in the UK, Jane Creaton, ‘Modernizing the courts and the legal profession’ (2003) 9 Contemporary Politics 115; Avrom Sherr, ‘Legal education, legal competence and Little Bo Peep’ (1998) 32 The Law Teacher 37.

^[18] Townes O’Brien et al (2011), above n 16, 56.

^[19] Vivien Holmes et al (2012) ‘Practising Professionalism: Observations from an Empirical Study of New Australian Lawyers’, accepted for publication in (2012) 15 Legal Ethics.

^[20] Townes O’Brien et al (2011), above n 16.

^[21] Graduate Careers Australia, Graduate Course Experience 2010 <http://www.graduatecareers.com.au/wp-content/uploads/2012/01/gcereport2010.pdf>.

^[22] See, eg, Nancy Falchikov and David Boud (eds), Rethinking Assessment in Higher Education: Learning for the Longer Term (Taylor and Francis e-Library, 2007); Roy Stuckey et al, above n 17, at Ch 7.

^[23] Nancy Falchikov and David Boud (eds), above n 22, Chapter 11.

^[24] See also Paul Maharg, ‘The culture of mnemosyne: Open-book assessment and the theory and practice of legal education’ (1999) 6 International Journal of the Legal Profession 219.

^[25] Erik Driessen and Cees Van Der Vleuten, ‘Matching Student Assessment to Problem-based Learning: Lessons from experience in a law faculty’ (2000) 22 Studies in Continuing Education 235, 236.

^[26] For a very small selection of the writing available see, eg, Roy Stuckey et al, above n 17, Ch 7; Paul Maharg, above n 26, 219–39; John Biggs, Using constructive alignment in outcomes-based learning, in John Biggs and Catherine Tang (eds) Teaching for Quality Learning at University (Society for Research into Higher Education, 3^rd ed, 2007), ch. 4; Per Lauvas et al, ‘Why This Inertia in the Development of Better Assessment Methods?’ (2000) 6 Quality in Higher Education 1, 91; Sue Swaffield, ‘Getting to the heart of authentic Assessment for Learning’ (2011) 18 Assessment in Education: Principles, Policy & Practice 433.

^[27] David Boud and Nancy Falchikov, ‘Aligning Assessment with long-term learning’(2006) 31 Assessment and Evaluation in Higher Education 399, 402.

^[28] Peter Knight, ‘Grading, classifying and future learning’, in Nancy Falchikov and David Boud (eds), above n 22.

^[29] There is large body of writing about SIMPLE. See, eg, M Hughes. et al, ‘SIMulated Professional Learning Environment — Programme Final Report’ (September 2008) <http://www.jisc.ac.uk/media/documents/programmes/elearninginnovation/simple_final_report.pdf> or Karen Barton et al, ‘Authentic fictions: simulation, professionalism and legal learning’ (2007) 14 Clinical Law Review 143.

^[30] Quote taken from a presentation by Paul Maharg and John Garvey, ‘Future Ed 2: Making Global Lawyers for the 21st Century. SCs [Standardised Clients] +SIMPLE’ <http://www.slideshare.net/paulmaharg/future-ed-2-making-global-lawyers-for-the-21st-century-scs-simple.

^[31] To accommodate the diversity of content area within the course, the number of students and size of the course, six main convenors look after specific practice areas, overseen by a PPC Convenor. Some 40–60 casual legal professional staff mentor students, either within the specific content areas or as practice mentors in practice management.

^[32] These roles are all ‘played’ by practising legal practitioners who are employed on a casual basis to provide authentic responses to students’ work as completed.

^[33] Quoted from the Professional Practice Course Outline provided to students for the Winter 2011 iteration of the course, ANU Legal Workshop 2011.

^[34] Description of the Professional Practice Core adapted from Course Outline 2011, above n 33.

^[35] Boud and Falchikov, above n 4, 406.

^[36] Adapted from the list provided by Boud and Falchikov, above n 4, 408–9.

^[37] Feed-forward is distinguished from feedback: it is provided on the basis that the students will use the information and ‘feed it forward’ into their revised work rather than just passively reading it.

^[38] Noting that we do not encourage our Senior Partners from being unprofessional in their variable habits, we wish to mentor the best of professional behaviour and not the worst (no matter how realistic the behaviour may be to practice).

^[39] Noting, of course, that some transactions may have pre-determined outcomes that need to be achieved in order to demonstrate competency — i.e. an incomplete conveyance is probably not a good indicator of competency, unless there are significant factors which preclude this outcome.

^[40] As suggested by Boud and Falchikov, above n 4, 409.

^[41] Boud and Falchikov, above n 4, at 410.

^[42] Stephen Darwin and Alex Knight, ‘Evaluation Report. Professional Practice Core: GDLP Integrated Learning Environment’, Semester One 2011, unpublished, 2.

^[43] Darwin and Knight, ibid, 3.

^[44] Stephen Darwin, Evaluation and Course Development Report Professional Practice Core: GDLP Integrated Learning Environment, Semester Two 2011, unpublished, 3

^[45] Ibid, 3

^[46] Ibid, 4.

^[47] Ibid, 6.

^[48] Ibid, 7-8

^[49] Ibid, 8

^[50] In 2010.

^[51] Darwin, above n 44, 4.

^[52] Driessen & Van Der Vleuten, above n 25, at 241