• Specific Year
    Any

Arup, Christopher --- "Research Assessment and Legal Scholarship" [2008] LegEdRev 3; (2008) 18(1&2) Legal Education Review 31

RESEARCH ASSESSMENT AND LEGAL SCHOLARSHIP

CHRISTOPHER ARUP*

On election, in 2008, the new Australian Government put a stop to the Research Quality Framework (RQF) which was gearing up for its first assessment round. Nonetheless, the Government has since announced that its place will be taken by a new research quality and evaluation system, the Excellence in Research for Australia initiative (ERA).1 ERA will depart from the RQF in certain respects and some useful comparisons can be made. However, the essential issue remains the nature of assessment and in particular the combination of publication metrics and peer review.

The aim of this article is to suggest how research assessment might have an effect on the intellectual approaches taken to legal scholarship in Australia. Part I notes the objectives of research assessment and situates them within the modern managerial approach to research. Part II sets out the RQF and ERA definition of research quality and considers how that definition might sit with the many different approaches to legal scholarship. Part III continues this theme, discussing in detail the selection of assessment measures and especially the design of metric measures such as research outlet rankings. Part IV finishes with some predictions regarding the advantages that might ensue to individuals and institutions from assessment.

I  RESEARCH ASSESSMENT

This part relates the Government’s objectives and considers the part research assessment might play in the management of research.

A Research Assessment Objectives

According to the Government’s Submission Specifications,2 the primary purpose of the RQF was to be assessment. Such a framework would reassure constituencies that research was being rigorously assessed through internationally recognised processes. It would allow research groups to be benchmarked both nationally and internationally. If that sounds rather disciplinary, the Specifications added that the RQF was:

underpinned by the desire to promote a vibrant research culture, encourage researchers to undertake high quality research and make research available to the Government, the higher education sector, industry and the wider community.3

The new Minister’s announcement takes a similar line. ‘The Commonwealth invests billions each year in research. The ERA model will provide hard evidence that taxpayers are getting the best bang for their buck in this critical area’. In terms of shaping research, the Minister foreshadowed that:

The ERA will build on work done to date in defining areas of strength, and will aid the development of our ‘hubs and spokes’ model for research infrastructure that is based on all universities having centres of excellence in specified fields.4

If the effect of the RQF or the ERA was to raise research quality across the board, it would be worth the considerable effort and law academics could embrace it warmly. Even if it stepped up competition, that would not be foreign to academics who, right from their success in school, have prospered by a meritocratic rating system. Instead, if there was to be a concern, it would be that the system had narrowing or channelling effects that actually undermined research quality. It would work against variety, innovation or collegiality in research. There would also be a fairness check: a query whether the system really did make judgements on the basis of merit. Overall, this article is not hostile to the idea of a research assessment exercise, but it takes the view there are some critical choices to be made, particularly about the nature of the measures that are to be employed to strike the ratings.5

B Modern Research Management

Once, the work of academics in the humanities and social sciences was funded almost entirely from salary. That salary included a component for time spent on research (roughly 30 per cent), together with regular sabbaticals and any extra time individuals put into their own research (‘after hours’). Those staff members who had tenure were left largely alone to choose their own projects. Research was still requisite, certainly to obtain promotion, and individuals took their own initiative to devise projects that would contribute to knowledge and woo publishers.

Starting with a Labor Government, under the direction of the Education Minister, John Dawkins, but stepping up considerably in the 1990s, the universities have been drawn into the public-sector management revolution.6 Combining neo-liberal economic principles with the new management sciences (especially accounting and human resource management), this system pursues goals, applies incentives, makes measurements, encourages competition and requires accountability. Productivity, excellence and relevance are among its key values. While positive in many respects, the regime does seem to withdraw the trust and autonomy that characterised the funding of the academic profession in earlier days.

This loss is keenly felt, at least by traditional academics, but arguably the effect is only partial. One recent commentary on legal education identifies three strands to ‘knowledge production’, as commodification, flexibilisation and segmentation.7 Of particular relevance is the idea of flexibilisation; that is, a growing stress on performative knowledge and experience, just-in-time problem-solving and work-based learning. It is argued below that legal research is tending in this direction. Nonetheless, Andy Boon, John Flood and Julian Webb warn against underestimating the strength of traditional academic values. They feel that law academics have retained discretion to make significant choices about subject matter, method and outlook.8

This remaining academic freedom cannot be quite the same as success in one or other facet of academic life, whether that be in terms of law school promotion, publication contracts, competitive grants, government appointments, law firm retainers or international consultancies. Such success is conditioned by the prevailing climate, though it is also a matter of how well the individual ‘plays the game’. Academics must juggle roles. They are salaried knowledge workers toiling in a changing production system and labour market for research. They are also energetic careerists and entrepreneurs in an expanding field of intellectual and social capital.9

Given such a mix of incentives and ambitions, it is likely a selective measurement system — one that favours a distinctive kind of quantity or quality, for instance — will have an influence on research.10 It might make less of an impact if it operates retrospectively and the academics subject to its regulation cannot be sure that it will not be changed again. Nonetheless, it would not be surprising if an assessment system engendered adaptive, and even strategising, behaviours, as well as some ‘perverse outcomes’. The precise nature of those effects might vary with the measures applied.

The recent stress on quantity has had its own effects. I would put aside the very real problem of encouraging and assisting those who do not write for publication at all, instead focusing here on those who are productive. The stress affects rate of publication, principally. However, style and format are influenced too, as publications in law begin to see, for example, work split into smaller bits, articles with multiple authors rather than sole-authored or co-authored pieces, and even different versions of the same thing.11

II QUALITY LAW RESEARCH

This part identifies the RQF and ERA definition of research quality. It endeavours to characterise the different approaches to legal scholarship in order to ask whether some approaches might be favoured by the definition.

The shift to quality seems a welcome relief from this feverish pace. Yet it, too, is likely to have its preferences. The RQF’s brief was to assess the quality and impact of research. ERA puts it slightly differently, specifying that its interest is in research excellence. All the same, the indicators it will pursue — indicators of research activity and intensity, research quality, and excellent applied research and translation of research outcomes — are very similar to the RQF.12 The main difference will be in its greater reliance on metrics over peer review as the method of assessing these attributes. The Government argues that the metrics on which it will rely, such as research grants and publications, already incorporate their own systems of peer review. All the same, ERA metrics are likely also to favour certain kinds of research over others.

Are the preferences identified at the level of definitions? The RQF Submission Specifications gave a definition of research quality. It was defined as referring to the quality of original research.13 This quality included its intrinsic merit and academic impact; academic impact in turn being related to the recognition of originality of the research by peers and its effect on the development of the discipline areas within the community of peers. While this definition had basic requirements, stressing new knowledge rather than derivative or imitative work, it was very much peer oriented and thus could conceivably incorporate the law discipline’s own idea of knowledge. The test would have come when the assessors rated the four best outputs that the universities had submitted on behalf of their nominated research groups, together with their full bodies of work for the census period.

This left quality very much to be determined by the RQF’s own review process, though the guidelines did say that peer review would be informed by metrics.14 In the case of law, these metrics were to be the rankings for journals and publishers. The metrics were not to include citations for publications, research income or higher degree by research (HDR) student loads and completions. Nevertheless, information about income and students was to be included in the context statement which the research groups submitted for assessment, and was presumably to be weighted in some degree when the panel gave each group its final rating between five and one.

In contrast, the ARC consultation paper does not offer a definition of research quality. Instead, consistent with the greater emphasis on metrics, it identifies indicators of research quality to include ‘publications analysis (ranked outlets, citation analysis and percentile analysis where relevant) and research income awarded on the basis of peer review’.15 It adds a category of research activity and intensity, where measures are to include research income, HDR student load and completions, and staff full-time equivalence (FTE) data. It suggests that HDR loads and completions can be a measure of research quality too. However, even though metrics are to play a bigger part in ERA, the use of indicators is to be combined with expert review by research advisory committees. Overall, the research has to be benchmarked as international, national, emerging or not competitive. Finally, there is a judgement to be made about the weight to place on the various metrics.

If much of the exercise is about defining quality, such assessments can present a more fundamental challenge for the law discipline. Because of the definition they give to ‘research’, some scholarly activities and outputs will be excluded from consideration altogether. The RQF specifications defined research as the creation of new knowledge and/or the use of existing knowledge in a new and creative way so as to generate new concepts, methodologies and understandings. This could include synthesis and analysis of previous research to the extent that it is new and creative.16

ERA adopts the same definition and it harks back to the definition employed in the Higher Education Research Data Collection (HERDC) process for deciding which outputs can be submitted to DEST for counting.

A Approaches and Methods

It is worthwhile making some observations about the diversity of legal research in order to suggest which might be favoured by an assessment exercise. The Council of Australian Law Deans (CALD) has been at pains to point out that legal research embraces a variety of approaches. Their 2005 statement lists 10 approaches: doctrinal, theoretical, critical/reformist, fundamental/contextual, empirical, historical, comparative, institutional, process-oriented and interdisciplinary.17 Are all such approaches likely to be regarded as evidence of quality research?

At one end of the spectrum is the traditional legal research characterised as doctrinal. Much of the writing in law continues to pursue, appropriately, the exposition of legal decisions. It is in search of a unified set of principles. Such so-called ‘black-letter law’ gives primacy to the text. It is this research of law texts that has been hardest to explain to researchers in other disciplines, and sponsors like DEST (now the Department of Innovation, Industry, Science and Research or DIISR). Progress has been made and the RQF Submission Specifications for Panel 11 offered some comfort in respect of the most problematic publications, student and practitioner books. Under the heading, ‘Acceptable Types of Research Outputs’, it states:

In Law, for example, Textbooks are often considered to be comparable in quality to Research Reports provided they incorporate significant scholarly research and contribute to legal knowledge. However, Textbooks that contain mainly primary sources will not usually be regarded by the Panel as acceptable Research Outputs.18

Such a proviso could be read as excluding only casebooks and annotated statutes. What, though, of the status of a textbook in a field that has already been mapped to some extent by others or a second edition of an established textbook? Both such books are a feature of law writing today. Yet the synthesis and analysis of judicial decisions, legislative material, administrative guidelines, government reports, international treaties and other scholarly writing remains valuable work, especially in a common law jurisdiction. It would be hard to think that Blackstone would not have made the RQF, though that might be putting the bar too high. Even today, the process of identifying and formulating ‘the law’ in many areas is sophisticated work because that law is to be found in a variety of places, it is often complex and it continues to change. If we can take heart from the view of the Research Assessment Exercise (RAE) Law Panel in the United Kingdom, it is not the use of the traditional legal method itself that is the problem. Rather, the individual work falls short if it goes over old ground and/or it lacks any critical analysis of the law.19

At the time of writing, the Australian Research Council (ARC) is still in consultations with the discipline groups such as law, so this kind of specification is not yet available for ERA. Because the ARC proposes to evaluate all research publications for all academic staff in each discipline cluster during the reference period, the issue of inclusion is magnified. In fact, the first critical point will be the administration of the specifications for HERDC because it is the publications collected here during the reference period that will be evaluated. Then it will be expressed through the rankings that the different journals and publishers receive.

Definitions are problematic not just for black-letter law but for other approaches that concentrate on the text. If we offer a new reading or interpretation of (the) law, does this add to legal knowledge? Comparing draft RQF Context Statements with a colleague from another discipline, the author suggested that many law academics specialise in ‘discourse analysis’. The colleague responded: ‘but not like cultural studies’. In fact, critical legal studies and postmodernist approaches have given a fillip to the study of the legal text and this approach is evident in Australian work.20

The approach we take to the text can be broadened out by calling it ‘law and policy’. Commonly, this approach remains one of exposition or articulation, unless it ventures (as socio-legal studies does) into explanations of the emergence of the law or its paths to implementation. Law and policy is more likely to add an evaluation to the exposition of the law. Usually, the standpoint for that evaluation is those values considered internal to law: meaning, authority, coherence and consistency. Law is evaluated again as a system of rules. This kind of analysis is worthwhile too. It is arguably where our expertise is strongest and the approach might be said to be confirmed by the social theory of Niklas Luhmann and Gunther Teubner.21 Law as a social system is cognitively open but normatively closed: it interprets communications from outside according to its own values. However, does this kind of evaluation, coupled perhaps with suggestions for rewriting of the law, qualify as research?

The study of legal policy uses external reference points too. In the European tradition, law lends itself to the canvassing of moral claims, such as natural rights arguments or appeals to notions of justice and fairness. The more recent American engagement with law has recommended the economic analysis of law according to its standards of efficiency and maximisation of wealth. It was taken up by law academics such as William Landes and Richard Posner.22 Some of us borrow its notions in an argumentative way. Yet, it has been pursued largely in economics journals away from law.23 It can be used abstractly to set up hypotheses and model legal rules, but is this approach any more ‘research’ than jurisprudence? It does make predictions about the social impact of the law, some persuasive, but often it is built on bold assumptions about human nature and social conditions. Would it more properly be considered ‘research’ to investigate the values of those who make or take the laws, to see if they are consistent with these philosophies, as historians, social anthropologists and political scientists would appear to do?

B Curiosity and Utility

Doctrinal research can be useful. If practitioners ‘know’ the law, they can work more effectively with it. Nonetheless, the Government might place at the other end of the spectrum the research that is interested primarily in how law can be applied to achieve social purposes.

If legal research is to be useful in instrumental terms, the law needs to be tested empirically through the investigations of the social sciences. Often this means taking a greater interest in legal practices and legal impacts. Did a new set of legal protections or powers promote economic and social improvements? How were the costs and benefits of the law distributed? It is a real challenge to answer questions like this. Legal researchers have to learn about social structures and action as well as legal artefacts. For instance, it is not easy to isolate the influences of the law from other social factors. Indeed, it is not easy to measure social outcomes at all. Often, if the inquiry is to bear fruit, it has to be narrowed down to less ambitious questions concerned more with description than explanation.

Some in the social sciences take the view that these kinds of questions can only be answered by using quantitative methods, which does not really suit many in law. Qualitative research is valuable. Indeed, it can give a richer view so long as it combines awareness of institutional settings with pursuit of particular practices — structure with agency.24 This method allows us to persist with documentary analysis, while casting the net for documents far wider than authoritative legal sources. We can add interviews with key actors (including lawyers and other intermediaries), as well as statistics, and even, where possible, coding for variables and applying statistical analyses to identify correlations and causation. Legal research has been moving in this direction. The competition for ARC grants has been one driver, most directly because this kind of research readily justifies the expenditure of money. In terms of its appeal, it looks most likely to guarantee the sponsor a concrete finding or a useful recommendation.

Nonetheless, the tradition of socio-legal studies tells us that quite often law cannot be treated as a ready tool. It is not only research into law as a means to an end that is worthy of support.25 Once, the concern was not enough instrumentalism; too much talk about law was rhetorical. Now could there be a danger of too much instrumentalism? As the CALD paper on legal research points out, finally, law cannot be subsumed within any other — more scientific — discipline.26 Assessors should realise that legal researchers are almost always going to give attention to the peculiarities of law, to think that the discourses and processes within law matter, even when they are also looking at law as an economic or social phenomenon. Sponsors want to see if and how law can produce an economic benefit or solve a social problem, or even just act effectively as an instrument of regulatory policy. Yet the role of law might be to provide a forum or a medium in which the nature of those benefits or problems is contested or a policy is resisted.27

III  RESEARCH PUBLICATION OUTLETS

This part identifies the roles for peer review and metrics in assessing quality research. Design problems are raised, in particular the difficulties of ranking journal and book publication outlets. Throughout, it considers how the choice of metrics might favour certain approaches to legal scholarship.

Would the RQF (now the ERA) drive legal research further in this direction, crowding out other approaches? Overall, the definitions of research and quality do not seem determinative.28 Rather, researchers will have to wait to see what peer review reveals about current notions of quality research. Given the direction that ERA is taking, they will also consider the role that metrics plays in channelling research. A crucial consideration is the kind of legal research that finds favour with the top ranked journals and publishers.

A Peer Review and Metrics

Peer review is sometimes accused of subjectivity. A visitor from the team reviewing the United Kingdom RAE referred to a tendency to rate the kind of research that the assessor does herself — or would like to do.29 This tendency could make the identity of the panellists (RQF) or committee members (ERA) an important factor. A concentration of power to define research quality is a concern, and Moodie is correct in saying: ‘Without a plurality of people, methods, and paradigms informing research funding, a country risks missing a radically new line of research’.30 The Australian Research Council chair Professor Margaret Sheil has indicated that steps will be taken to prevent small groups of reviewers becoming too influential when the systems for awarding research grants and judging research quality converge within the Council.31

Peer review also has a workload challenge. The RQF Law Panel faced a daunting task reading all the best quality outputs that would have been nominated, let alone having to make an appraisal of the full bodies of work for each researcher in the six year census period (2001–2006). Certainly, much work would have to be let out to assessors. Perhaps the work would have been ‘audited’ — that is, sampled in some way — rather than read comprehensively. The full body of work would not have been read.

Potentially, the workload for ERA is even higher, for it aims to evaluate all the research publications of all the academic staff in each discipline cluster during the reference period. Metrics is a way to control workloads. The ARC consultation paper says that peer review of a sample of outputs may be required

where there are no appropriate indicators for the discipline or information about the outputs is not captured by the indicators being used for the discipline (e.g. the majority of research outputs are not indexed by the citation data supplier).32

High-volume decision making systems are drawn to measures that appear objective. Often these measures are quantitative. Necessarily, perhaps, such measures are not sensitive to the individual case. They are more likely to be surrogate measures and rules of thumb than exact measures.33 Would the RQF (now the ERA) metrics be mere proxies for quality? The focus here is on the ranking of research publication outlets. Ranking involves: (a) deciding which kind of outlets will be counted at all; (b) establishing a hierarchy of outlets (journals versus books and so on); and (c) ordering a particular type of outlet in tiers (eg, top-tier journals through to the also-rans).

B Journal Rankings

The focus so far has been on ranking journals. The drive within DIISR is for single journal ranking systems — in the shape of a pyramid, with only a very few journals in the top tier (percentages). In the Monash University Faculty of Business and Economics, most disciplines are comfortable with this approach. In contrast, the CALD debated the request for such a ranking system at length during 2007.34 At the time, the majority of deans opposed any ranking system, while a minority thought it would be more realistic for the discipline to put something to DIISR. This year, CALD has undertaken to compile a list. However, events have overtaken it because DIISR has proposed that the index of the United States university law school Washington and Lee, be adopted, subject to any amendments that might be accepted. This index is problematic for Australian law academics because it is highly US-focused, and strong submissions have been made against using the Washington and Lee index by entities such as the Australasian Law Teachers Association.35

In disciplines like econometrics or the hard sciences, it would seem the aim is to publish one’s recent findings in the most prominent journals. In the leading international journals, submissions are sent to multiple referees and they receive harsh scrutiny. However, they are not unique in that respect. Finally, they are top ranked because most academics do want to get their articles into them and, consequently, they have a high rejection rate. This competition might drive up quality.

Is there a nucleus of journals in law like this? What determines the prestige of a title or ‘high impact’ in the law discipline? Law academics can agree that there are well established journals — journals associated with prestigious law schools both local and abroad. The prestige of local schools would likely place such reviews as the Melbourne University Law Review, Federal Law Review, Sydney Law Review and Monash Law Review (all very good journals) in the small top tier.

Should the ranking system go behind the title to look at quality control and who edits and publishes in the journals? One complication for the law discipline has been the practice of student editors, at least for law school reviews in the United States and Australia. This peculiarity has been smoothed out somewhat, with faculty advisers taking a more active role and articles routinely being sent to external referees. Nonetheless, outsiders should be alert to the hazards — for example, that some United States law schools reviews might defer to very senior contributors, especially when the status of the contributor is a factor in the ranking of the journals themselves.36

Is a more fundamental problem with the reviews that they are associated with particular schools? If law academics were consulted, there might be found a general sense that some journals carried more substantial pieces than others or at least that they had less trouble obtaining their quota of solid contributions. It is another matter whether they always publish the very best of research. Who, then, would rank the journals independently if they were to be based on the prestige of the title? It would be a difficult task for a group of law deans collectively and, naturally enough, each university might be inclined to favour its own review.

Furthermore, many of the different fields of law boast their own subject-specific journals now. To ensure they reach their colleagues, academics who have committed to a particular field tend to place their articles in these field-based journals. A single ranking system would not only need to order the law school reviews, it would have to rank the generalist law school reviews against such subject-specific law journals as well. It is important to note that the ERA indicator will not just highlight a few ‘top-tier’ publications. It will rank most other publications as low or despatch them to an also-ran category of not ranked.37

In other disciplines, the top journals are often edited by experienced academics. This is so in law too, but some of the field-based journals in law are edited instead by practitioners — for instance, because the journal is produced by an industry association. As a result, some of the law journals devoted to particular fields of law have had good practitioner editors. Some such law journals, while they, too, largely carry articles by academics, are published by commercial publishing houses aiming to make a profit. Such origins do not reflect badly on the academic rigour of the journals. Due in part to DIISR type requirements, most journals are refereed now, or they will not receive submissions from academics.

Another division, though, is between journals published locally and journals published overseas. At this time, many universities are also endeavouring to raise the international profile of their research. Again, it is fair to say that those Australian academics committed to a field tend to publish in the specialist journals that enjoy an international circulation, rather than, say, Australian, American or United Kingdom law reviews. Yet these international journals might not attract the same ranking as the best general reviews either. The proposed Washington and Lee index is instructive in this regard. It favours reviews and, largely, this means the United States law school reviews. Some other reviews and some specialist subject-matter journals are represented, but they are placed well down in the rankings.38 It is important to note that the United States law school reviews are foreign-based journals, not truly international journals. Traditionally, they have been quite parochial in their subject matter and they even fail to cite other relevant work if it is published outside the United States.

Other biases are evident too. For example, the La Trobe University journal Law in Context was established to provide a more theoretical perspective on the study of law and to recommend the use of methods from other disciplines to cast light on legal practices. It has lost some of that difference now that many more law academics are widely read and versatile in their approaches.39 Yet, even if this was not the case, it would be unlikely, on the basis of law school prestige, for Law in Context to go into a small top tier. In the Washington and Lee index, even Law and Society Review and Law and Social Inquiry, which are both edited from the United States, are well down in the rankings despite their obvious academic rigour.

The general point is the lack of an obvious hierarchy in law. If the focus is on who publishes where, the finding is that law academics spread their work around. In keeping, the United Kingdom RAE Law Panel observed that

First-rate articles were found in both well-known journals and relatively little-known ones. Conversely, not all the submitted pieces that had been published in ‘prestigious’ journals were judged to be of international excellence. These two points reinforced the RAE Panel’s view that it would not be safe to determine the quality of research outputs on the basis of the place in which they have been published or whether the journal was ‘refereed’.40

If metrics have their uses, submission and rejection rates would seem to offer more objective data than perceptions of prestige. Such rates would have to depend on reliable figures being forthcoming from the journals themselves. Even so, such numbers can have more than one meaning too. For example, high rejection rates could mean the journals are receiving a lot of low quality submissions, rather than a host of good quality papers vying for space. In the United Kingdom, Kevin Campbell, Alan Goodacre and Gavin Little recommend that rejection rates be combined with the ratings of the researchers who submit the papers.41 They use the first United Kingdom RAE ratings to do this. On this basis, outputs would have to be subject to peer review, the first time around at least, before the system could rely on journal rankings.

What would be a way around these issues? If academics accept the idea that they are all swimming in the same pool, the short answer might be a simple accord. Collectively, the system strikes a compact on a hierarchy of journals — almost any hierarchy would do. Thus, by fiat, certain journals would become the top journals and law academics could now compete to be published in them. Yet, surely these rules of the game would need to be prospective: it would not seem right, if the stakes were high, to make them the measure retrospectively.

Would it matter which journals were placed in the top tiers? The generalist reviews are willing to publish a range of legal topics. Moreover, it is fair to say that, if they still tend towards the traditional approaches to legal research, these reviews do accommodate other approaches today. The main sticking point for acceptance in these reviews would be Australian material. CALD argued strongly that jurisdiction-bound research could be world class. Yet a generalist review overseas, such as the Harvard Law Review, might not be prepared to publish an article that contained exclusively Australian material, whereas the subject-specific journals overseas appear sufficiently interested to do so.

Is the practical answer for academics to tailor their research to the inclinations of the journals? For example, academics from small countries can ensure at the least that they include some comparative or international law research in the work they submit? The new forms of globalisation suggest that legal research must be conducted at many levels and in many places, internationally, nationally and locally.42 Might there be a curious bias, even a cultural cringe, if the reference point had to be United States law? And even if the rankings sent this message, would academics change their practices? Who do academics write for: research assessment, the legal academy and future employers, each other in an academic intellectual property community, or legal practitioners and policy makers? Some researchers also write for more mixed audiences with an interest, for example, in science policy, international trade, development studies, or culture and the arts. And, while many would publish in journals based in the common law countries, some are trying to reach an audience in other parts of the world. Such journals are likely to be left out of a law list entirely. Finally, articles can take a significant period of time to go through the writing and refereeing process, and by the time of publication, the criteria for assessing research quality (such as the rankings) may have changed.

C Book Publications

Another dilemma is provoked by the lesser importance that the law discipline has attached to journal publication compared to other disciplines. Law academics publish books. Frequently, they are cases and materials or textbooks, but law academics also publish research monographs and chapters in edited collections of papers. The concern is that book publishing will increasingly defer to the publication of journal articles.43

A research assessment system discriminates against certain types of books. If it does not exclude them from consideration by definition, it discounts them in a hierarchy of outlets or makes it uncertain whether individually they will receive a high rating. Publishers are apprehensive that RQF- and ERA-type systems will discourage textbook writing. At the moment, the practice looks lively and Australia enjoys a healthy competition in most areas of law. Perhaps, independent of a research assessment system, such publications will continue to serve purposes that the law schools can support. For their part, textbooks earn a little revenue. They also advertise to students and practitioners that the teachers in a particular school are learned in the law. Often they represent a genuine interest in teaching and learning and an effort to convey knowledge more effectively.44

The risks are not confined to texts. Finding a publisher for a research monograph is not easy, given the economics of the publishing industry. An original manuscript may be written over a long period, without guarantee of commercial publication. At the 2006 ALTA Conference, the sympathetic Federation Press publisher, Chris Holt, remarked on the irony. The Government was stressing research quality just as it was getting harder to find a market for monographs, certainly for those dedicated to Australian studies.45 New technology might relieve the situation a bit, with its capacity for cheap production and short print runs, even for e-books that are printable on demand.

Book publishing also presents a ranking challenge. Would those rankings be based on prestige, the refereeing process or the difficulty of obtaining a contract? Like journals, the outlets are diverse. Australian textbook publishing has been dominated by two publishers, now part of very big conglomerates. Nevertheless, the scholarly university presses have been developing law lists and a number of other publishing houses have become active in the field, such as Kluwer, Ashgate, Edward Elgar, Hart and, in Australia, Federation Press. One will also find interesting law books with other publishing imprints — Routledge/Cavendish, Cameron May or Earthscan, for instance.

D Other Metrics

This discussion has concentrated on publication rankings. Metrics includes other measures such as citation. Citation counts can cut through the publication rankings issue by going straight to citations for individual papers, though citation rates can also be used as a means to rank the journals. For the first round of the RQF, the law panel specifications eschewed citations analysis.46 The ARC consultation paper says that ERA will consider a number of indicators of research quality, with a particular focus on research publications and bibliometrics, including ‘profile of citations against relevant Australian and worldwide benchmarks where relevant and available’ and ‘centile analysis of publications against most highly cited world papers where relevant’.47 The institution will be required to submit a unique identifier for each publication indexed in the citation database(s) during the reference period.48

The basic question is what citations tell us about quality. Can we say that legal knowledge is built up the same way as the sciences might be, from one finding about the natural world to another? Even in other disciplines, the motives for citing are queried.49 Citation rates are also affected by the accessibility of the original work, which takes us to the issue of the range of library holdings and even the bibliographic databases. There are implementation issues such as collection of citations. Many journals are accessible electronically now, making searches of their citations easier, provided the publishers give their permission. Citation indexes are well developed in other disciplines, though not without controversy regarding their criteria, coverage and transparency.50 DIISR has commissioned the very proficient Austlii team to work on a system of collection for law.51 However, would the system only be able to take those citations that had been made in local reviews and journals? The citations that are contained in books are especially hard to collect. In the law discipline, academics read books, but books are not so easy to find from electronic databases and certainly to read online. Academics rely on catalogues and inter-library loans to reach them. There are signs in the ARC consultation paper that ERA will collect citations only from ‘certain journal sets’.52

While the RQF law panel specifications said that research income was to be included in the context statement for the research group submission, they did not treat it as a ‘metric’. The ARC consultation paper signals that research income will be among the measures of research activity and the indicators of research quality for ERA.53 Research income will be collected according to the HERDC categories 1–5.54 Competitive grants and other income have not been pervasive features of research in the law discipline the way they have in the sciences or business and economics. Nonetheless, it would seem appropriate to recognise the success that some law academics have had in recent years with competitive peer-reviewed ARC grants. In law, though, the spread of these grants has been quite narrow, so the metrics should largely act to identify some very good researchers, predominantly in the Group of Eight (Go8) universities. There are also important pockets of category 2-4 income, sourced predominantly from the public sector. Making research income a metric would intensify the effort to obtain this income, with its own impact on the kind of research that is done. If the ARC favours research promising a discrete, practical result, it is likely to push it towards the instrumental end of the spectrum (see above). The RQF impact measure would have done the same and, possibly now, ERA’s interest in applied research and the translation of research outcomes.

ERA makes HDR student loads and completions a measure of research activity and intensity. Overall, HDR numbers are not as high in law as they are in many other disciplines. At least their measurement will acknowledge the research intensity within certain law schools where PhD enrolments are strong.

IV  IMPACTS

Such systems have consequences for the distribution of rewards and advantages throughout the sector. This part considers the possible impacts on the futures of individual law academics and the fortunes of the many law schools.

A Law Academics

The clearest consequence for the individual academics is the ratings they receive. If the assessment is important to the institution, then those individuals who rate highly will be rewarded, subject to the flexibility that institutions have to vary, for instance, teaching and administrative loads, remuneration, or promotion criteria. Such reward would seem to be a goal of the system. The concern is felt among those who might not rate highly.55

In the New Zealand system, all academics submit to a rating.56 Like the RAE, the RQF’s distinction was to be more subtle. Assessment was to be on the basis of research groupings. The universities themselves, through their faculties and departments, were to decide which individuals were to be included in the groups. Internal selection would turn mainly on the anticipated quality of the individual’s outputs. However, this discretion made the selection subject to the research profile the institution wished to adopt. Conceivably, individuals could be left out because their research did not fit the profile in terms of field or approach.

In the sciences, teamwork is common, but law is often a solitary pursuit. While individuals specialise, it is still common for them to write in more than one area of law. The RQF Submission Specifications for Panel 11 recognised this practice. The groupings did not necessarily have to cohere around a common research agenda, except in the broadest sense. It was enough that the members had law in common.57 Moreover, the minimum number for a group was to be five researchers. On this basis, few law schools but the largest could propose a group dedicated, for instance, to a specific area of law research. Furthermore, it was not necessarily good for collective morale or reputation to single out certain areas of law. While some law schools were proposing multiple groupings based on subject matter, others were contemplating submitting a single law group.

ERA will now confirm this strategy because it is interested in institutional performance at the level of ARC discipline clusters overall. Furthermore, it will evaluate the research activity and quality of all staff in that discipline and not just those who are nominated by the institution. This means, though, that it will seek a means to show what proportion of staff are research active in any one institution. The consultation paper also says that institutions will have the opportunity to obtain breakdowns of the evaluations by individual research and academic units.58

B Law Schools

What might be the consequences for the institutions overall? In the case of the RQF, the results were to direct a share of the Government’s funding to universities for research infrastructure (IGS) and HDR student support (RTS). The ARC consultation paper now says that ERA will not determine the allocation of research block grants.59 In any case, law schools earn most of their income from teaching. Arguably, the highest stake is reputation. Reputation is an intangible, though it can have a major impact on postgraduate student enrolments, fields for appointments to staff, and success with competitive grants and commissions for research.

In the United Kingdom, the RAE has been a reason why some law schools actively recruit academics with quality outputs on their resumes.60 The RQF had devised specifications to safeguard against schools ‘gaming’ the system and picking up academics just to bolster their groups for the submissions or claiming academics only nominally on their books. Nonetheless, provided the researchers were coming into a three-year position at a level of 0.4 FTE or above, the RQF allowed them to bring with them the publications they had researched at another institution. The ARC consultation paper has left this issue open for the time being, though it does say that the ARC Advisory Council prefers an approach that attributes publications based on a researcher’s institutional affiliation at the census date rather than the institutional affiliation based on HERDC; that is, where the publication activity occurred.61 All other indicators will definitely be collected according to the institutional affiliation of the activity (such as the grant).

‘Gaming’ the system has to be distinguished from the shifts between institutions that create research concentrations and critical mass in research, either in law generally or perhaps in subject-specific fields. Such groupings might well be an intended consequence of a research rating system. Recruiting high performers is an obvious response. Moreover, such competition can have cumulative effects as well. For example, senior researchers and good infrastructure attract grants and higher-degree research students. Grants fund research fellowships to work full-time on research and writing for publication. Sometimes the fellowships release senior staff from teaching and administrative duties. Increasingly, together with studentships, they bring in excellent new staff who complete at least the initial research and writing for co-authored publications.62 A few of the larger schools are a long way down this path already, while some of the smallest schools may have to stick with the model of solitary research. However, all law schools will be reformed somewhat if research methods move closer to the science model.

Behind this trajectory is the thinking that research benefits from groupings within the one institution (critical mass, cross-fertilisation, economies of scale and scope) and that a small country can really only be expected to support a small number of excellent research schools. In this way, ratings levels (five-star, four-star and below) or benchmarking distinctions (international, national, emerging and non-competitive) would be another factor at work in the shake-out of Australian law schools — a shake-out that has been looming since the numbers of schools swelled so much.

Other factors are at work too, such as the freedom to charge full fees if the market will bear it. At the end of this road is the prospect of an accreditation and stratification system that is based on resource and performance measures like the American Bar Association system in the United States.63 While prestige varies, formally speaking all law schools are equals in Australia. Academics should expect basically the same working conditions whichever school they are appointed to. That equality has frayed around the edges in terms of, for example, teaching loads and support services, but each school can claim to do the same thing, and the Council of Australian Law Deans have treated issues like the RQF and ERA gingerly, precisely to avoid fuelling divisions in the ranks.

The RQF specifications did allow for cross-institutional combinations and it would have been interesting to see if any such groups were nominated. Certainly, at the moment in law, collaboration is healthy, with cross-institutional co-authored publications and ARC project teams. The specifications also allowed co-authored publications to be nominated among the best quality outputs and the Context Statement was meant to indicate ways the group had fostered collaborative research. However, the fundamental dynamic of the RQF was towards competition between institutions. The ARC consultation paper also says that it is important that ERA encourages collaboration across institutions.64 Cross-institutional outputs can be submitted by each institution involved provided they are appropriately identified. It is unclear at this stage how they will be weighted within each institution’s evaluation.

A division may emerge between the schools that educate graduates for elite national and international commercial work and those that educate for the local small business, household, criminal and public-interest practices. These latter schools would be advised to concentrate their limited resources on research that is related to these local areas of practice — subject-specific but also applied and engaged with the community in its approach. However, even such strategies are problematic. The United States experience suggests that the top human rights positions are fiercely contested65 and that the public sector jobs are also attractive to certain graduates from the elites, especially to women graduates.66 Not surprisingly, a proportion of students from the second-tier schools want the chance to compete for jobs in the commercial law sector. Thus it can be said that students who lack social capital are disadvantaged across the board.

If the thinking behind research concentration is basically sound, policy analysis should still balance the consequences for the rank-and-file schools if a select few become research elites. Those other schools in the ruck would still be expected to meet the basic requirements of a respectable law school.67 The current professional requirements (the eleven areas of knowledge all Australian States accept as compulsory for admission to practise – the Priestley 11) mean that all schools must teach credibly across a spectrum of private and public law subjects. Furthermore, once graduates enter the profession, they want to be able to move across the sector, for example between private and public practice, and between practice and other law work.68 It would seem that both firms and employees make ‘mistakes’ about who is fitted to the demands of the work or that they change their preferences. In this labour market, it is better that the students receive a good generalist education, with transferable skills, rather than be streamed too early.

Would not the task of the rank-and-file schools be made even harder if they face an operational cycle of staff turnover — for instance, seeing young stars develop their profile and then leave for an elite institution? If it is accepted that teaching should be research-based, these schools might have trouble keeping honours and postgraduate students. However, just as transfers are inevitable, it is likely that the rank-and-file will always keep some good researchers. Decisions about moving (labour market preferences) certainly turn on the nature of employers, but other factors come into play as well, such as family needs (partners, children and parents), housing affordability and social amenity. These are human factors that a bureaucratic or market system does not readily concede.

More to the point, perhaps, is the treatment of the students in the rank-and-file schools. It might be more important to the quality of legal services if these students have exposure to teachers who are also scholars and researchers than that the standards of the elites increase even more. How can Australia get universities into the international rankings while ensuring the base is sound too? Of course, these dilemmas only lead into larger questions about education policy: how much to spend on law compared to other disciplines, on university graduates compared to technical training or basic literacy.69

V  CONCLUSIONS

It is certainly hard to argue with quality and one should not overstate the impacts that research assessment might have. The law discipline has been expanding, due largely to the sustained student demand for places and the increase in schools offering the professional degree. In many areas of the law, research is strong in Australia. While it is not easy for academics to get started, good new academics are emerging through both the teaching and research assistant routes.

The concern would be if any system undermined the diversity and wisdom the discipline currently displays. Research assessment schemes are part of a rationality of public sector management. They are designed to make academics more competitively and instrumentally minded. While a research quality assessment system would stimulate some worthwhile research (that otherwise might not have appeared), the uncertainty is whether it will discourage the kind of long-term investment and collaboration that builds real scholarship in the discipline. It is likely that the safeguards lie in the design of the system and not in its resistance altogether. The big shifts in the sector will be driven by less pliable and more ruthless forces.


[*] Professor of Business Law, Faculty of Business and Economics, Monash University, Melbourne, Australia. The views expressed here are personal rather than institutional.

[1] Kim J Carr, Minister for Innovation, Industry, Science and Research, Commonwealth Government, ‘New ERA for Research Quality’, (Press Release, 26 February 2008) <http://www.minister.innovation.gov.au/Carr/Pages/neweraforresearchquality.aspx> at 8 December 2008.

[2] Department of Education, Science and Training (DEST), Commonwealth Government, Research Quality Framework: Assessing the Quality and Impact of Research in Australia, RQF Submission Specifications, (September 2007) [1.2].

[3] Ibid.

[4] Carr, above n 1. See also Australian Research Council (ARC), Consultation Paper, Excellence in Research for Australia (ERA) Initiative (June 2008) 5 <http://www.arc.gov.au/pdf/ERA_ConsultationPaper.pdf> at 8 December 2008.

[5] Of course, the stresses can vary. For a comparative study, see Dominic Orr, ‘Research Assessment as an Instrument for Steering Higher Education — A Comparative Study’ (2004) 26 Journal of Higher Education Policy and Management 345.

[6] Simon Marginson and Mark Considine, The Enterprise University: Power, Governance and Reinvention in Australia (2000). Lately, see Carole Kayrooz, Gerlese S Akerlind and Malcolm Tight (eds) Autonomy in Social Science Research: The View from United Kingdom and Australian Universities (2007).

[7] Andy Boon, John Flood and Julian Webb, ‘Postmodern Professions? The Fragmentation of Legal Education and the Professions’ (2005) 32(3) Journal of Law and Society 473.

[8] Ibid. For example, in the intellectual property field, academics can still choose whether to write about intellectual property history or recommendations for improvement to current laws, or whether to be more supportive or critical of intellectual property protection. See Staniforth Ricketson, ‘Intellectual Property as a Field of Research and Scholarly Inquiry: Jim Lahore as the Pioneer’ (2005) March Intellectual Property Forum 10.

[9] Bryant G Garth and Yves Dezalay, The Internationalization of Palace Wars: Lawyers, Economists and the Contest to Transform Latin American States (2002); Margaret Thornton, ‘The Idea of the University and the Contemporary Legal Academy’ [2004] SydLawRw 24; (2004) 26 Sydney Law Review 481.

[10] Linda Butler, ‘Modifying Publication Practices in Response to Funding Formulas’ (2003) 12(1) Research Evaluation 39; Colin Steele, Linda Butler and Danny Kingsley, ‘The Publishing Imperative: The Pervasive Influence of Publication Metrics’ (2006) 19(4) Learned Publishing 277.

[11] See, eg, Tony Sheil, ‘Beauty of Small but Perfect Forms’, Higher Education Supplement, The Australian (Sydney), 5 March 2008, 26.

[12] ARC, above n 4, 5.

[13] DEST, above n 2, [1.7].

[14] DEST, above n 2, [6.0].

[15] ARC, above n 4, 7.

[16] DEST, above n 2, [1.6].

[17] CALD, Statement on the Nature of Legal Research (May and October 2005) <http://www.cald.asn.au/resources.htm> at 8 December 2008.

[18] DEST, above n 2, [11.2].

[19] The RAE Law Panel has been both encouraging and disparaging of law texts: ‘In many of the student and practitioner texts submitted the Panel was pleased to find a freshness and depth of rigorous analysis (for example, in reconciling or distinguishing cases, in interpreting or applying statutes or in examining areas in the light of relevant policies) that, in its view, amply justified a rating of international excellence. However it was in relation to textbooks, particularly practitioner books but sometimes also those aimed at students, that the Panel occasionally felt a sense of disappointment. In some cases it was clear that the author(s) had gone to enormous pains to set out the law in a very clear exposition, but essentially the exposition was one of the ‘known world’ rather than of new material, telling the reader little that was not already known, or its coverage of new material was merely descriptive’. See ‘RAE 2001 Overview Reports from the Panels’, Unit 36, Law (2001) [5] <http://195.194.167.103/overview> . at 11 October 2008).

[20] Consider, for example, Patricia Loughlan’s recent discussion of the uses of language like piracy and theft: ‘Pirates, Parasites, Reapers, Sowers, Fruits, Foxes ... The Metaphors of Intellectual Property’ [2006] SydLawRw 12; (2006) 28 Sydney Law Review 211.

[21] See, eg, Gunther Teubner, ‘After Legal Instrumentalism? Strategic Models of Post-regulatory Law’ (1984) 12 International Journal of the Sociology of Law 375.

[22] See, eg, William M Landes and Richard A Posner, The Economic Structure of Intellectual Property Law (2003).

[23] Ove Granstrand (ed), Economics, Law and Intellectual Property: Seeking Strategies for Research and Teaching in a Developing Field (2003).

[24] Pierre Bourdieu, The Social Structures of the Economy (2005). For the lessons specifically for legal research, see Deborah R Hensler, ‘Reflections on the Use of Empirical Research in Legal Policy Reform’ [2003] NewcLawRw 1; (2003-4) 7 Newcastle Law Review 1.

[25] Brian Z Tamanaha, Law as a Means to an End: Threat to the Rule of Law (2006).

[26] CALD, above n 17.

[27] For an approach to the study of law that endeavours to accommodate that complexity and fluidity, see Sol Picciotto, ‘Regulatory Networks and Global Governance’, (Plenary Paper given at the WG Hart Legal Workshop, London, 27-29 June 2006) <http://eprints.lancs.ac.uk/232/1/Reg_Networks_ & _Glob_Gov.pdf> at 8 December 2008.

[28] It is true that a research quality exercise would not necessarily be hostile to a critical, contextual approach to legal studies. See Margaret Thornton, ‘The Dissolution of the Social in the Legal Academy’ (2006) 25 Australian Feminist Law Journal 3, 17. But see Margaret Thornton, ‘The Retreat from the Critical: Social Science Research in the Corporatised University’ (2008) 50(1) Australian Universities Review 5.

[29] See Sir Gareth Roberts, Review of Research Assessment: Report by Sir Gareth Roberts to the Funding Bodies (May 2003) <http://www.ra-review.ac.uk/reports/roberts.asp> at 8 December 2008. The United Kingdom Government wants the system to pursue more metrics-based measurement after the RAE 2008 is completed.

[30] Gavin Moodie, ‘Potential Flaws in Funding Mechanism’, Higher Education Supplement, The Australian (Sydney), 5 March 2008, 26.

[31] Bernard Lane, ‘Experts Will Be Balanced’, Higher Education Supplement, The Australian (Sydney), 5 March 2008, 22.

[32] ARC, above n 4, 7.

[33] A classic study is Jerry L Mashaw, Bureaucratic Justice: Managing Social Security Disability Claims (1983).

[34] CALD also established an RQF working group and the author is very grateful to its members, especially Professor Kathy Bowrey, for the benefit of their expert analyses. Of course, the opinions expressed here are entirely those of the author.

[35] DIISR has expressed a preference to continue with the Washington and Lee list. In response, CALD formed a subcommittee to determine the amendments it wished to see. CALD consulted widely with law school deans, a panel of academics and international experts. At the time of writing (October 2008), DIISR had not released an amended list.

[36] For the United States, see Robert M Jarvis and Phyllis Coleman, ‘Ranking Law Reviews by Author Prominence – Ten Years Later’ (2007) 99 Law Librarian 573; Ronen Perry, ‘The Relative Value of American Law Reviews: A Critical Appraisal of Ranking Methods’ (2006) 11 Virginia Journal of Law and Technology 1.

[37] See table in ARC, above n 4, 16.

[38] See Washington and Lee University School of Law, Law Journals: Submissions and Rankings <http://lawlib.wlu.edu/LJ/index.aspx?mainid=89> at 8 December 2008.

[39] Jeremy Webber, ‘Legal Research, the Law Schools and the Profession’ [2004] SydLawRw 27; (2004) 26 Sydney Law Review 565; Richard Collier, ‘“We’re All Socio-Legal Now?” Legal Education, Scholarship and the “Global Knowledge Economy” — Reflections on the UK Experience’ [2004] SydLawRw 25; (2004) 26 Sydney Law Review 503; Fiona Cownie, Legal Academics: Culture and Identities (2004) ch 3.

[40] RAE Law Panel, above n 19, [4].

[41] Kevin Campbell, Alan Goodacre and Gavin Little, ‘Ranking of United Kingdom Law Journals: An Analysis of the Research Assessment Exercise — 2001 Submissions and Results’ (2006) 33 Journal of Law and Society 335.

[42] Boaventura de Sousa Santos and César A Rodríguez-Garavito (eds), Law and Globalization Below: Towards a Cosmopolitan Legality (2005).

[43] Kevin Campbell, Douglas W Vick, Andrew D Murray and Gavin Little, ‘Journal Publishing, Journal Reputation and the United Kingdom’s Research Assessment Exercise’ (1999) 26 (4) Journal of Law and Society 470.

[44] Similarly, the edited collection may become a labour of love for the law discipline. It might also have a life as the outcome of a workshop or conference that has been undertaken as part of a competitive grant.

[45] Chris Holt, ‘Researching Law’ (Plenary address given at the Australasian Law Teachers Association Annual Conference, Victoria University, Melbourne, July 2006).

[46] DEST, above n 2, [1.18].

[47] ARC, above n 4, 8.

[48] Ibid 10.

[49] Tim Birkhead, ‘Beware the Grant-getting Lottery’, Higher Education Supplement, The Australian (Sydney), 6 June 2007, 33.

[50] See, eg, Bernard Lane, ‘Research Review Heats Up’, Higher Education Supplement, The Australian (Sydney), 23 January 2008, 21, discussing the Thomson ISI index.

[51] Australian Legal Scholarship Library, Australasian Legal Information Institute <http://www.austlii.edu.au/austlii/research/2008/lief/> at 8 December 2008.

[52] ARC, above n 4, 12.

[53] Ibid 7.

[54] Ibid 9. More recent indications are the collection will be confined to Category 1.

[55] Subject to the uncertainties about the measures of assessment changing, individuals can aim prospectively for better ratings. For a study of such impact, see Ameen Talib, ‘Behavioural Consequences of the Research Assessment Exercise in England’ (2002) 15 Accounting Research Journal 186.

[56] For information about the impact of the New Zealand system (on education academics), see Noeline Alcorn, Russell Bishop, Carol Cardno, Terry Crooks, Peggy Fairbairn-Dunlop, John Hattie, Alison Jones, Ruth Kane, Patricia O’Brien and John Stevenson, ‘Enhancing Education Research in New Zealand: Experiences and Recommendations from the PBRF Education Peer Review Panel’ (2004) 39(2) New Zealand Journal of Education Studies 275. For ratings results, see Tertiary Education Commission, ‘Performance-Based Research Fund: Evaluating Research Excellence, the 2006 Assessment’ (2007); see especially Table A-24: ‘Subject-Area Results — Law’, 144.

[57] DEST, above n 2, [11.1].

[58] ARC, above n 4, 15.

[59] Ibid 5.

[60] Birkhead, above n 49.

[61] ARC, above n 4, 11.

[62] The RAE Law Panel also commented that: ‘Many younger scholars had made good use of their postgraduate research and had written specialised work of very high quality’: RAE Law Panel, above n 19, [3]. In Australia, new staff members of smaller schools who do their own PhDs externally, with the larger schools, will bring the benefit of their postgraduate publications to the smaller schools. Likewise, some of the research fellows move on to academic appointments in the smaller schools.

[63] External factors such as ‘international trade in educational services’ might also drive us to an accreditation system. See Laurel S Terry, ‘The Bologna Process and its Implications for U.S. Legal Education’ (2007) 57 Journal of Legal Education 237. For instance, if Australian degrees are to receive recognition so that graduates may sit the United States state bar examinations without obtaining local United States qualifications, an accreditation system might become necessary, see Laurel S Terry, Carole Silver, Ellyn Rosen, Carol A Needham, Robert E Lutz and Peter D Ehrenhaft, ‘Transnational Legal Practice (International Legal Developments in Review: 2007)’ (2008) 42 International Lawyer 833, at fn 93.

[64] ARC, above n 4, 7.

[65] Bryant G Garth, ‘Noblesse Oblige as an Alternative Career Strategy’ (2004) 41(1) Houston Law Review 93.

[66] Ronit Dinovitzer and Bryant G Garth, ‘Lawyer Satisfaction in the Process of Structuring Legal Careers’ (2007) 41 Law and Society Review 1, 43. In Australia, there is no systematic study like the United States ‘After the JD’ project but the experience of competition for jobs with the personal injuries firm, legal aid and prosecution offices is in keeping. See Christopher Arup, ‘The Regulation of Human Capital: Public Frameworks and Firm-based Policies’ in Christopher Arup, Peter Gahan, John Howe, Richard Johnstone, Richard Mitchell and Anthony O’Donnell (eds), Labour Law and Labour Market Regulation (2006), 648.

[67]This would be the case unless the demands become so competitive that some fail to keep accreditation, amalgamate, or become ‘pre-law’ preparatory schools under a graduate model of professional education (note the new Melbourne model). Rationalisation might result from a university sector restructure if the Bradley Committee recommends radical changes, see Review of Australian Higher Education: Discussion Paper, Commonwealth of Australia, June 2008, 19. The Review’s report is expected in December 2008.

[68] Arup, above n 66.

[69] Richard Florida, The Flight of the Creative Class: The New Global Competition for Talent (2005).