• Specific Year
    Any

Edmond, Gary; Cunliffe, Emma; Martire, Kristy; San Roque, Mehera --- "Forensic Science Evidence and the Limits of Cross-Examination" [2019] MelbULawRw 23; (2019) 42(3) Melbourne University Law Review 858

FORENSIC SCIENCE EVIDENCE AND THE LIMITS OF CROSS-EXAMINATION

Gary Edmond,[1]* Emma Cunliffe,[1]**

Kristy Martire[1]† and Mehera San Roque[1]

The ability to confront witnesses through cross-examination is conventionally understood as the most powerful means of testing evidence, and one of the most important features of the adversarial trial. Popularly feted, cross-examination was immortalised in John Henry Wigmore’s (1863–1943) famous dictum that it is ‘the greatest legal engine ever invented for the discovery of truth’. Through a detailed review of the cross-examination of a forensic scientist, in the first scientifically-informed challenge to latent fingerprint evidence in Australia, this article offers a more modest assessment of its value. Drawing upon mainstream scientific research and advice, and contrasting scientific knowledge with answers obtained through cross-examination of a latent fingerprint examiner, it illuminates a range of serious and apparently unrecognised limitations with our current procedural arrangements. The article explains the limits of cross-examination and the difficulties trial and appellate judges — and by extension juries — experience when engaging with forensic science evidence.

Contents

I Introduction

[Cross-examination] is beyond any doubt the greatest legal engine ever invented for the discovery of truth. ... [C]ross-examination, not trial by jury, is the great and permanent contribution of the Anglo-American system of law to improved methods of trial procedure.[1]

The adversarial trial process is predicated on the efficacy of cross-examination as an ‘engine ... for the discovery of truth’.[2] As Wigmore’s quote demonstrates, common law scholars have not been shy about ascribing near mystical powers to skilled cross-examination. This article offers a critical reassessment of the trust that orthodox legal doctrine places in the efficacy of cross-examination, specifically in respect of the challenge of identifying and conveying significant problems with forensic science evidence.

In this article, we build on a growing body of scholarship about these problems, including work that documents the failure of legal processes to identify and counter problems with forensic science evidence. In order to illustrate how cross-examination fails to serve its supposed function as an engine for truth, we present a case study: an unsuccessful challenge to the probative value of latent fingerprint evidence. Our study — based on R v JP[3] and (on appeal) JP v Director of Public Prosecutions (NSW)[4] (collectively

JP’) — suggests that cross-examination is not necessarily effective in influencing fact-finders’ assessment of the probative value of expert opinions in criminal proceedings. This is so even when, as in this instance, the cross-examination was well-prepared and directed at forensic science testimony that was exaggerated, misleading and failed to incorporate or address relevant scientific knowledge.

The frailties of cross-examination are not, of course, the only structural impediment to accurate fact determination in cases involving forensic science evidence. The ensuing analysis should be read in the context of weak admissibility standards, limited resourcing (especially for the defence), and increasing reliance on so-called trial safeguards such as rebuttal witnesses, admissibility compromises and judicial instructions. We have previously written, separately and collectively, about each of these problems.[5] However, our analysis of the failures of cross-examination is important because it is inconsistent with orthodox legal commitment to the efficacy of cross-examination as an ‘engine’ with the tendency to make criminal proceedings fair and the state, through its expert witnesses, accountable.

This article explores the difficulties encountered by defendants, and their (usually publicly-funded) lawyers, when attempting to demonstrate non-trivial limitations with forensic science evidence adduced by the state.[6] Rather than rely exclusively on the interactions — the questions and particularly the responses — from the trial (and the voir dire), our analysis juxtaposes responses from the transcript with scientific research and authoritative scientific pronouncements. This method allows us to contrast the testimony from the proceedings with what is known about forensic science evidence in academic, scientific and policy realms beyond the courtroom. We consider these academic, scientific and policy perspectives to be the benchmark of institutional knowledge regarding the forensic sciences. Our approach enables the reader to observe conspicuous differences between the trial evidence and this benchmark, as well as the inability of participants, including the ‘expert’ witness and judicial officers, to appreciate their significance.[7]

The issues canvassed in this article are particularly important for non-DNA feature comparison procedures, such as latent fingerprints, ballistics, tool marks, handwriting and documents, shoe, tyre, paint, hair and fibre, soil, image, gait and voice comparison evidence. Mainstream scientific knowledge is rarely presented to the trier of fact in criminal prosecutions where evidence derived from these procedures is adduced and relied upon by the state.[8]

On the relatively rare occasions when methodological and other epistemo-logically destabilising issues are raised, our experience — which resonates with the course of events in the case study — suggests that it is unlikely that those issues will be understood or taken seriously.[9] Further, judicial inter-vention at trial — most likely to manifest through the provision of directions and instructions on evidence and the burden of proof — is unlikely to assist with problems. Such interventions are rarely informed by scientifically-based insights or methodological sophistication. Rather, they tend to elide and obs-cure gaps between mainstream scientific research and conventional legal app-roaches to expert evidence.

For reasons explained in this article, our system of accusatorial trial has not managed scientific evidence well. This observation is particularly impor-tant given the increased state reliance on forensic science evidence in recent years. We argue that the criminal legal system’s stubborn persistence with traditional adversarial processes tends to blind judicial officers to knowledge, restricts the provision of feedback about accuracy and system performance, and thereby impedes institutional learning. Indeed, institutional inertia, hierarchical deference, technical illiteracy and institutional biases seem to make it difficult for judicial officers to recognise or respond to fundamental epistemic problems with forensic science evidence.

A A Case Study and a Few Caveats

Our decision to focus on a single case permits us to demonstrate some of the failures suggested in the broader literature, but it simultaneously imposes methodological limitations. It might fairly be said, for example, that this

case is not representative of contemporary litigation. This section therefore explains both our selection of JP and the reliance that we place upon it in the ensuing analysis.

First, and perhaps most importantly, in Australia as in cognate juris-dictions, there have been relatively few challenges to (non-DNA) forensic science evidence, including latent fingerprint evidence.[10] For a multitude of reasons, including resource constraints and a skills deficit at the criminal bar, methodologically sophisticated challenges to forensic science evidence are exceptionally rare. The infrequency of serious challenges does not reflect the validity and reliability of procedures (or methods) in regular use by the state. To the contrary, following a multi-year review of the forensic sciences, the National Research Council (‘NRC’) of the United States (‘US’) National

Academy of Sciences (the ‘Academy’), in its Strengthening Forensic Science in the United States: A Path Forward report (‘NRC Report’), concluded:

With the exception of nuclear DNA analysis, however, no forensic method has been rigorously shown to have the capacity to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.[11]

It is the exceptional nature of JP — a rare challenge to the long venerated ‘science of fingerprint examination’[12] — that makes this case both informative and revealing.

The case of JP is illuminating because, more than a century after the introduction of latent fingerprint evidence, it represents the first sustained attempt to contest such evidence — both the procedures used and the strength of conclusions drawn — in an Australian criminal proceeding.[13] JP is the first substantial challenge to the underlying procedures of fingerprint identification despite longstanding legal reliance, massive expansion of its usage, and technological developments. This relatively late challenge to latent fingerprint evidence comes on the heels of a series of recent authoritative scientific reviews (such as the NRC Report) that question fingerprint examiners’ methodology, expression of results, and routine exposure to domain-irrelevant information (such as the identity or criminal history of the suspect).[14] Few of these issues had previously been raised or recognised in legal processes, despite routine reliance on latent fingerprint evidence as definitive proof of identity for more than a century.

Close consideration of JP reveals ‘gaps’ between mainstream scientific knowledge and the information presented in expert reports and courtroom testimony. These gaps usually pass unrecognised within criminal litigation. Further, JP suggests that even when gaps are exposed, many forensic scien-tists, lawyers, and judges seem incapable of recognising their evidentiary implications for the case and beyond.[15] This case is significant precisely because scientifically-informed challenges to routine forensic science evidence are unusual. Close attention reveals structural impediments to recognising, and taking seriously, issues that are central to understanding the probative value of the expert opinion — notably, attending to validity, scientific reliability, uncertainty, error, human factors, impartiality and the way the opinion is expressed.[16] The case clearly illustrates some of the difficulties encountered when actors within the legal system attempt to engage with, let alone understand, relevant scientific research and advice. Simult-aneously, it casts light on conventional legal strategies and explanations employed by prosecutors, expert witnesses and judicial officers to resist the implications of inconvenient scientific knowledge.

Our case is susceptible to analysis because it proceeded to trial, where it was decided by a judicial officer obliged to provide ‘reasons’.[17] The trial decision was appealed, such that another judicial officer reviewed the verdict and the reasoning. Although trials and appeals are not uncommon, our criminal justice system depends overwhelmingly on guilty pleas.[18] This structural reliance on pleas makes a case involving a well-prepared challenge to forensic science evidence even more of an outlier. A trial and

appeal, and the semi-public records they generate, enable us to access and

consider issues that routinely pass unrecognised in both plea negotiations

and contested proceedings. The explicit challenge to the reliability of fingerprint identification evidence within the case affords a very rare oppor-tunity to evaluate how courts engage with issues that are not usually disclosed or considered.

Critics might try to defend cross-examination by denigrating the performance of the publicly-funded lawyer in this particular trial (and appeal). It might be said that any failings with the cross-examination were the result of inexperience or the particular tactics employed.[19] We believe such interpretations are difficult to sustain. By juxtaposing scientific knowledge and recommendations with extracts from the proceedings, we intend to allow the transcript to speak for itself. For those defending the status quo, recourse to inexperience or tactical mistakes as some kind of justification or explan-ation for failure in well-prepared challenges has system destabilising impl-ications.[20] How often, we might wonder, is the primary trial safeguard ineffective or less effective than it ought to be? And, what are those who defend the status quo doing to address weak performances? Moreover, what are the ‘appropriate’ tactics in JP, and why should guilt or innocence depend on such subtleties? We also wonder about the tactical genius of the large number of very experienced legal practitioners (some now senior judges) who did not raise, and presumably did not appreciate, the very serious methodological problems with forensic science evidence they seemed to have accepted at face value or raised perfunctorily in circumstances where clients protested their innocence.[21] Defence of prevailing practices perpetuates ign-orance and inconsistency in a system that purports to be fair and consistent in its dispensation of justice.

Finally, it is not our intention to engage in ad hominem criticism of the expertise and sincerity of the latent fingerprint examiner who appeared in JP. This particular latent fingerprint examiner appears to have been selected

to testify because he was the senior examiner in the region encompassing Dubbo, where the offence and prosecution took place. We accept that he can analyse and compare fingerprints in ways that are superior to lay persons.[22] We do not contend that his testimony is insincere. The witness frankly acknowledged that he was unfamiliar with a variety of scientific materials and all of the opinions he expressed in response to the challenge to his evidence might have been sincerely held. There are few reasons to believe that the testimony reported below is not representative of perspectives that are widely held by members of his community.[23]

Our primary concern is with the question of whether testimony proffered by forensic practitioners, called and relied upon by the state, provides decision-makers with accurate information and the ability to evaluate it.[24] Do those called by the state, and recognised as experts by the courts, testify in terms consistent with what is known on the basis of scientific research about latent fingerprint examination? In the terms of our evidence law, are their opinions based on ‘knowledge’?[25] Much of the testimony reproduced below demonstrates: a lack of knowledge; indifference to scientific research, methods and advice; and a preference for longstanding assumptions, beliefs, commitments and impressions pervasive among latent fingerprint examiners. This preference may be difficult to reconcile with the goals, rules and needs of the criminal legal system.

It is precisely here where our system seems to fail — because, in the case of JP, the decision-makers did not seem to understand that scientific knowledge undermined to some degree the probative value of the fingerprint evidence.[26] To be clear, it is not that latent fingerprint evidence is without probative

value, but rather that the opinion is less conclusive than is suggested by the prosecutor, the fingerprint examiner and, in the end, the judicial officers.

In a prosecution that turns on latent fingerprint evidence, this observation required serious consideration in relation to the burden and standard of proof. Our case study is both exceptional and helpful because it brings these issues to the fore.

II Adversarial Trials, Trial Safeguards and

Cross-Examination

In the adversarial tradition, courts allocate primary responsibility for assem-bling and presenting evidence to the parties.[27] In accusatorial proceedings, the state bears the burden of proving the elements of a criminal offence beyond reasonable doubt. In theory, the state, through prosecutors, has a special obligation to present ‘credible evidence ... firmly and pressed to its legitimate strength but it must also be done fairly’.[28] While the defence has various obligations (which have been expanding over time), in practice, its actions are typically responsive.[29] Resource constraints, along with pervasive beliefs about the accuracy of forensic science evidence, place considerable pressure

on defendants and their publicly-funded lawyers to enter an early guilty plea, or limit challenges to forensic science evidence to cross-examination at

trial.[30] Most criminal defendants (and their lawyers) do not have access to an expert witness, or even expert advice, when considering their plea or preparing a defence.

For the tiny proportion of cases that make it to trial, legal rules and proc-edures are intended to make proceedings fair and outcomes both consistent and accurate. When expert evidence is presented at trial, admissibility

rules and procedures, such as cross-examination, are considered to be appropriate and implicitly effective ways of testing expert opinions. The following statement, from an influential US Supreme Court decision on

the admissibility of scientific evidence, conveys this pervasive common

law attitude:

Vigorous cross-examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but admissible evidence.[31]

Like many of the so-called trial safeguards, cross-examination is conven-tionally feted.[32] Its value to the truth-seeking function of trials is widely asserted, including when contested expert opinion evidence is admitted. The reality is often remote from such lofty valorisation. Cross-examination can be a very effective tool; however, even a well-prepared cross-examination may fail to serve its intended purpose. And, of course, in a context in which the state’s forensic evidence is rarely tested, it is worth observing that tools can only be effective to the extent that they are actually used.

Judges rarely reflect on some of the practical constraints limiting the cross-examination of experts. In addition to the actual opinion, it is contingent upon: (1) the ability and resources available to the cross-examiner; and (2) the knowledge, experience, honesty, resilience and persuasiveness of the witness; and (3) decision-makers appreciating the significance — however complex or methodologically subtle — of questions and answers, concessions, re-calcitrance and ignorance. In many cases, and perhaps most cases involving forensic comparison procedures, these contingencies tend to assist the prosecutor and the state, thereby contributing to the disadvantages confronting the frequently impecunious and technically-illiterate defendant.[33] For, where one or more of these contingencies ‘fails’, the resulting risk of misuse of evidence falls disproportionally upon defendants.

A review of empirical studies, focused on the ability of cross-examination to expose weaknesses in expert opinion evidence, concluded:

[S]tudies have found little or no ability of cross-examination to undo the effects of an expert’s testimony on direct-examination, even if the direct testimony is fraught with weaknesses and the cross is well designed to expose those weaknesses. ... [I]t is unlikely that defence cross-examination ... will reduce the impact of the forensic expert witness’s direct testimony.[34]

Similarly, a report prepared by the Law Commission for England and Wales questioned conventional commitment to trial safeguards:

Cross-examination, the adduction of contrary expert evidence and judicial guidance at the end of the trial are currently assumed to provide sufficient safeguards in relation to expert evidence ... However, ... it is doubtful whether these are valid assumptions.[35]

This article offers an empirical instantiation of these laboratory studies and reviews. Our case study demonstrates that, rather than operating as an engine for exposing weakness and uncovering truth, the impact of cross-examination is inconsistent and often banal.[36] Faith in cross-examination may blind

legal participants to limitations with opinion evidence and system constraints. Nevertheless, the ability to ‘test’ incriminating expert opinion evidence through cross-examination remains one of the primary justifications

for liberal approaches to the admission of ‘shaky’ incriminating opinion evidence, as well as confidence in the fairness of criminal proceedings and the safety of verdicts.

III A Brief Overview of Latent Fingerprint

Jurisprudence in Australia

[I]t may be the safest of all evidence.[37]

Latent fingerprint evidence is admissible and regularly admitted in Australian courtrooms.[38] It was first admitted at the beginning of the 20th century and was quickly accepted as sufficient to support proof beyond reasonable doubt where the question of identity was in issue.[39] From its origins in the Anglo-American criminal justice system — and predating DNA profiling and studies of forensic feature comparison procedures (eg ballistic, bite mark, hair, fibre, shoe, voice, foot and tyre comparisons) — latent fingerprint examination has equated the ‘matching’ of two fingerprints with the positive identification of a specific person.[40] Categorical identification, also known as ‘individualisation’, was sometimes supplemented with the locution to the exclusion of all others.[41] Positive identification is based on a pervasive commitment to fingerprints being both unique and permanent, and the related contention that when two fingerprints match, they were produced by the same finger. Significantly, the issues in JP concerned the probative value of the match decision — that is, how should match decisions be reported, and what does a match decision mean in relation to proof of identity?

Given what follows, it might be considered disconcerting that there have been so few serious challenges to the validity and scientific reliability of latent fingerprint evidence. We encourage readers to search the law reports and legal databases for themselves.[42] The dearth of epistemologically-informed challenges is revealing because the first ever independent validation studies of the procedures used by latent fingerprint examiners were conducted after 2009, in response to criticisms expressed in the NRC Report.[43] This dearth is interesting because the basic procedure employed by contemporary latent fingerprint examiners (Analysis, Comparison, Evaluation and Verification, known as ACE-V) was described as early as 1959.[44] While some of the methods used to locate, develop, document and analyse prints have been refined, comparison of latent fingerprints continues to depend on subjective interpretations by human examiners.[45] As technical practices and equipment changed during the course of the 20th century, fingerprint examiners did not rigorously evaluate their new procedures, develop probabilistic means of expressing results, or attempt to manage risks to their subjective analyses.[46]

Simultaneously, lawyers did not seek information, nor actively challenge traditional assumptions or new techniques and interpretations. The use of algorithms to search rapidly expanding electronic databases did not generate jurisprudence that addressed their implications for practice or the way ‘match’ opinions ought to be expressed. This is curious because some developments, such as the increasing use of databases, introduced new risks of adventitious matches, as the number of prints, and the number of similar-looking prints, rapidly increased. Moreover, many of the developments were occurring as courts were beginning to grapple with how biologists, population geneticists and statisticians might present the results of DNA profiling in probabilistic terms.[47] Legal credulity contributed to the persistence of prescientific and non-scientific practices among latent fingerprint examiners and the police organisations that host them.[48]

The few reported challenges in Australia, including a couple of excl-usionary decisions, are not concerned with methods, validity, reliability, accuracy, error or bias. Rather, they are focused on: the way reference prints were collected; latent fingerprint examiners transgressing legal boundaries (by offering opinions beyond identity, for example the age of a print or what a person was doing when a print was deposited); not providing images of the prints; and, not identifying specific points of similarity.[49] Others are focused on the failure of the trial judge to explain that a latent fingerprint matched to a particular person might have had an innocent origin.[50] None of the Australian decisions call into question the underlying methods, the idio-syncratic way latent fingerprint examiners equate a match with positive iden-tification, or the departure of practices from mainstream scientific research, methods and advice.[51]

More cautious and qualified concerns about fingerprint evidence, expressed in earlier decisions at the beginning of the 20th century, have gradually been effaced.[52] A decade ago, in a judgment endorsed on appeal, the Chief Justice of South Australia indicated that there was ‘no suggestion that the process of comparison ... is not a recognised and appropriate process’.[53] The reliability, even practical infallibility, of fingerprint evidence seems

to have been accepted, and promoted, by the courts through their accommodating practices. Components of a perverse feedback loop —

admissibility, legal reliance, the lack of successful challenges and

convictions — were all used as proxies for reliability to lend epistemic legitimacy to the field.[54] Oriented to the courts, latent fingerprint examiners substituted legal recognition and reliance for scientific insight into actual abilities and limitations.[55]

Finally, it is useful to locate the exchanges in JP in the context of Australian admissibility jurisprudence. We do not contest the admissibility of opinions about latent fingerprints in some form.[56] However, no Australian court requires, or has required, evidence of validity or demonstrable ability as part of its expert opinion admissibility practice.[57] Few Australian criminal courts require details about procedures, assumptions and reasoning.[58] Legal interest has focused on relevance, training and experience, perceived value, past legal practice and so forth.[59] Rather than provide a prophylactic against unreliability, misrepresentation and exaggeration, our admissibility juris-prudence relies heavily on trial safeguards available to the defendant and

the abilities of decision-makers. Weak admissibility jurisprudence places a considerable burden on codes of conduct for expert witnesses and cross-examination at trial.

IV Introducing JP

JP was charged with an aggravated break, enter and steal, which occurred in Dubbo on 4 October 2014.[60] The elderly occupants of a house woke to find a person in their bedroom rifling through drawers and items on the bedside table. The intruder was described by one of the occupants as young and ‘of teenager height’, and by the other as 15–17 years of age, wearing dark clothing.[61] No other description of the offender was provided.

A number of prints were recovered from the scene, including a palm print taken from an exterior surface and a partial print lifted from a jewellery box located in the bedroom.[62] It is not clear from the record how JP came to be a suspect.[63] However, a full set of prints was taken when he was interviewed on 8 October 2014. Based on a ‘match’ decision, JP was subsequently charged with the offence and remanded in custody.[64]

The matter was heard in the Dubbo Children’s Court commencing on

13 January 2015. The defence challenged both the late service and admissibility of the Expert Certificate (or report)[65] provided by a latent fingerprint examiner — a Detective Sergeant with the New South Wales Police Force (the ‘NSW Police’).[66] In his Expert Certificate, dated 18 December 2014, the examiner describes comparing the prints recovered from the scene with the fingerprint impressions taken on 8 October 2014.[67] He ‘identified’ the latent print on the jewellery box ‘to the Left Thumb of [JP]’.[68] The police prosecutor accepted that the fingerprint evidence was, in effect, the only evidence against JP and that without it the prosecution must fail.[69] The Magistrate rejected the defence applications, and the hearing proceeded with the Expert Certificate admitted into evidence. The latent fingerprint examiner was cross-examined across the three days of the hearing (13, 15 and

27 January 2015).

At the conclusion of the hearing on 27 January 2015, the Magistrate agreed that the only evidence capable of identifying JP as the offender was the fingerprint evidence.[70] Accepting the conclusion that the thumbprint was deposited by JP, and discounting discrepancies between the eyewitness desc-riptions and the appearance of JP, he found the offence proven.[71]

JP appealed his conviction to the Supreme Court under pt 5 of the Crimes (Appeal and Review) Act 2001 (NSW) by way of summons filed on 5 May 2015.[72] The grounds included a challenge to: the admissibility of the latent fingerprint evidence (grounds 1 and 6); the adequacy of the reasons for the decisions to admit the evidence and find JP guilty (grounds 2 and 4); the Magistrate effectively delegating decision-making responsibility to the latent fingerprint examiner (grounds 5 and 8); and that the evidence was insufficient to prove guilt beyond reasonable doubt (ground 9).[73] The appeal was heard in Sydney on 6 October 2015. Beech-Jones J granted leave to raise grounds 6 and 8, but rejected all other grounds and dismissed the summons.[74] He further refused leave to allow the defence to raise matters relating to s 135 and s 137 of the Evidence Act 1995 (NSW), since there had been no objection at trial.[75]

V Comparing Expert Testimony with Scientific Knowledge

In this section it is our intention to juxtapose the testimony received in JP with authoritative scientific reviews of latent fingerprint evidence — our ‘benchmark’. This enables us to observe how scientific knowledge, even when raised, may not influence legal decision-making. Indeed, in this particular trial — unusual because the procedures and the ability to make a positive identification were ‘contested’ — the latent fingerprint evidence was presented in a way that did not incorporate, or even acknowledge, fundamental scientific research and derivative recommendations. Instead, the dogma of fingerprints as effectively irrefutable evidence of identity was espoused and accepted. The judicial officers observed nothing that raised either a reasonable doubt or warranted public expression of concern.

The following sub-sections draw upon reports produced by the NRC,[76] the US National Institute of Standards and Technology (the ‘NIST’), and the Scottish Fingerprint Inquiry (the ‘SFI’). (The NIST produced the Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach report (the ‘NIST Report’),[77] and the SFI produced the The Fingerprint Inquiry Report (the ‘SFI Report’).[78]) These reports, and the associated inquiries, were conducted and published in the decade before the trial and appeal. The NRC Report was prepared by a subcommittee of the

Academy following a request and appropriation from Congress.[79] The Academy was established during Lincoln’s presidency, is independent from government, and is one of the world’s most prestigious scientific organ-isations.[80] The NIST is a US federal agency responsible for, among other things, metrology and the provision of standards.[81] These reviews were dominated by scientists, engineers and statisticians. The SFI, on the other hand, was a public inquiry overseen by a senior judicial officer follow-

ing mistaken identifications by a Scottish fingerprint bureau.[82] The reports, particularly the seminal report by the NRC, exposed a limited research base and openly questioned the value of many forensic science procedures.

A Introduction to ACE-V

To begin, it is useful to provide a brief introduction to ACE-V, the procedure used by latent fingerprint examiners in Australia, the US, Canada and the United Kingdom (‘UK’), among others. The following description is taken from the NRC Report:

The technique used to examine prints made by friction ridge skin is described by the acronym ACE-V: ‘Analysis, Comparison, Evaluation, and Verification.’ It has been described in forensic literature as a means of comparative analysis of evidence since 1959. The process begins with the analysis of the unknown friction ridge print (now often a digital image of a latent print). Many factors affect the quality and quantity of detail in the latent print and also introduce variability in the resulting impression. ...

...

If the latent print does not have sufficient detail for either identification or exclusion, it does not undergo the remainder of the process (comparison and evaluation). These insufficient prints are often called ‘of no value’ or

‘not suitable’ for comparison. Poor-quality known prints also will end the examination. ...

Visual comparison consists of discerning, visually ‘measuring,’ and comparing — within the comparable areas of the latent print and the known prints — the details that correspond. The amount of friction ridge detail available for this step depends on the clarity of the two impressions. The details observed might include the overall shape of the latent print, anatomical aspects, ridge flows, ridge counts, shape of the core, delta location and shape, lengths of the ridges, minutia location and type, thickness of the ridges and furrows, shapes of the ridges, pore position, crease patterns and shapes, scar shapes, and temporary feature shapes (eg, a wart).

At the completion of the comparison, the examiner performs an evaluation of the agreement of the friction ridge formations in the two prints and evaluates the sufficiency of the detail present to establish an identification (source determination). Source determination is made when the examiner concludes, based on his or her experience, that sufficient quantity and quality of friction ridge detail is in agreement between the latent print and the known print. Source exclusion is made when the process indicates sufficient disagreement between the latent print and known print. If neither an identification nor an exclusion can be reached, the result of the comparison is inconclusive. Verification occurs when another qualified examiner repeats the observations and comes to the same conclusion, although the second examiner may be aware of the conclusion of the first. ...

... Note that the ACE-V method does not specify particular measurements or a standard test protocol, and examiners must make subjective assessments throughout.[83]

B Scientific Method? ACE-V, Standards and Subjectivity

1 Authoritative Scientific Findings and Recommendations

The NRC Report and the NIST Report provide independent reviews of ACE-V. Both openly questioned its value as a rigorous (scientific) method, as well as its ability to support positive identification of suspects and eliminate errors:[84]

ACE-V provides a broadly stated framework for conducting friction ridge analyses. However, this framework is not specific enough to qualify as a validated method for this type of analysis. ACE-V does not guard against bias; is too broad to ensure repeatability and transparency; and does not guarantee that two analysts following it will obtain the same results. For these reasons, merely following the steps of ACE-V does not imply that one is proceeding in a scientific manner or producing reliable results. A recent paper by Haber and Haber presents a thorough analysis of the ACE-V method and its scientific validity. Their conclusion is unambiguous: ‘We have reviewed available scientific evidence of the validity of the ACE-V method and found none.’[85]

Subsequent research, undertaken after 2009, established the foundational validity of latent fingerprint comparison.[86] This relatively limited research base also provided, for the first time, an indication of the frequency

of errors.[87]

The NRC and the NIST accepted that fingerprint comparison performed using the ACE-V process offers a potentially valuable means of assisting with identification. They were, however, vitally concerned with the validity of the procedure, the performance of examiners, and accuracy. The committees responsible for these reviews insisted on the need for rigorous standards around quality and sufficiency of prints, match decisions, verification, documentation and so forth. Standards provide important safeguards because of the subjective nature of latent fingerprint comparison:

At every step in the ACE-V process, human factors can affect the outcome. ... Subjectivity is an inextricable part of the process. In the Analysis phase, for example, accurate identification of the characteristics that make prints of value depends on the examiner’s knowledge, training, and experience. Likewise, in the Comparison phase, variable factors, such as the elasticity of skin and uneven pressure, mean that there will never be perfect congruence between two prints, even if they originate from the same source. The examiner must resolve the question of whether there is sufficient agreement ‘within tolerance.’ ... [T]he examiner at least implicitly relies on a sufficiency threshold to resolve that question, and in setting this threshold, the examiner draws on professional knowledge and experience. There is little research at present that provides objective metrics for determining these tolerances.[88]

The lack of meaningful standards and its implications for practice were canvassed in the NIST Report. On the threshold for deciding whether a latent fingerprint is ‘sufficient’ for comparison, for example:

The Working Group found no research that explicitly addresses utility or sufficiency in the context of latent print analysis. ... Opening the box to study the process of judgment in every phase of ACE-V would provide the empirical foundation from which to develop best practices for each part of the process.[89]



2 Responses to the Cross-Examination in JP

Now we turn to compare these assessments with the testimony in JP. Consider the following exchange:

Defence Counsel: Do you agree that because the ACE-V technique depends on your capability as a human being to make observations and make subjective decisions that it is actually vulnerable to a number of sources of error?

Fingerprint Examiner: Again if ACE-V is done correctly and by a person in the right mind and with the correct tools and apparatus I don’t believe that there would be an error.[90]

At trial, ACE-V was described not only as effective, but effectively error-free. In this response, the subjective nature of the process is elided, as the (not entirely responsive) answer asserts that procedure and mindset provide safeguards against error.

Elsewhere, the witness acknowledged some of the subjective aspects of his analysis, comparison and evaluation:[91]

Defence Counsel: Is it predominantly the case that you’re using your own eyes to make observations and then draw conclusions from what you perceive?

Fingerprint Examiner: Yes with the — also in the software there are the ability to put marker points on the impressions as well and then yes it is just with my own eyes.

Defence Counsel: I’m going to suggest to you that a conclusion that a pair of impressions come from the same source depends on the ability of the examiner to [analyse] and compare the impressions is that right?

Fingerprint Examiner: Yes that is correct.[92]

Such concessions sit awkwardly against the contention that ACE-V is effectively error-free.

C Error and Error Rates

1 Authoritative Scientific Findings and Recommendations

Consider now the NRC’s conclusions about error — specifically in relation to latent fingerprint examiners applying ACE-V:

Errors can occur with any judgment-based method, especially when the factors that lead to the ultimate judgment are not documented. Some in the latent print community argue that the [ACE-V] method itself, if followed correctly (ie, by well-trained examiners properly using the method), has a zero error rate. Clearly, this assertion is unrealistic, and, moreover, it does not lead to a process of method improvement. The method, and the performance of those who use it, are inextricably linked, and both involve multiple sources of error (eg, errors in executing the process steps, as well as errors in human judgment).[93]

All subjective, or judgment-based, procedures are vulnerable to error.[94] Because errors are a feature of all processes involving humans, and part-icularly those involving human interpretation, the NRC recommended that latent fingerprint examiners — and other forensic scientists — should take immediate steps to measure and disclose uncertainty and accuracy:[95]

All results for every forensic science method should indicate the uncertainty in the measurements that are made, and studies must be conducted that enable the estimation of those values. ...

... [T]he accuracy of forensic methods resulting in classification or individualization conclusions needs to be evaluated in well-designed and rigorously conducted studies. The level of accuracy of an analysis is likely to be a key determinant of its ultimate probative value.[96]

Among its many virtues, measuring error and uncertainty enables examiners to improve their individual performance, to collectively standardise and to refine procedures.[97] It also enables them to provide decision-makers with the means of evaluating match opinions. Only when we are provided with an empirically-based indication of how often examiners make mistakes (in conditions resembling, though not necessarily identical to, those encountered in the case) are we in a position to rationally attribute a probative value or weight to their opinions.[98]

In discussing error, it is helpful to introduce a more recent report prepared by the US President’s Council of Advisers on Science and Technology (the ‘PCAST’), titled Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods (the ‘PCAST Report’).[99] This particular report, published after the trial and appeal, was not available to the parties in JP. Nevertheless, this report explicitly builds upon, and adds further authoritative support to, the conclusions reached by the NRC and the NIST. The PCAST Report was produced, at President Obama’s invitation, by some of the most eminent scientists in the US. It documents the results of a comprehensive review of the scientific research, that had been performed by late 2016, in order to assess seven forensic feature comparison procedures, including latent fingerprint comparison.

The PCAST Report endorsed the NRC Report, including its concerns about ACE-V.[100] In its review of research undertaken since 2009, the PCAST Report concluded that properly trained latent fingerprint examiners possess genuine expertise.[101] Studies confirmed that latent fingerprint examiners can match and discriminate between prints, and have a tendency to err on the side of finding prints not to match (false negatives), rather than falsely matching non-matching prints (false positives).[102] These findings presumably come as a relief to both fingerprint examiners and legal institutions in the aftermath of the NRC Report and the NIST Report. The PCAST Report was not, however, a complete endorsement of the claims and practices of the latent fingerprint community. For, in addition:

PCAST finds that latent fingerprint analysis is a foundationally valid subjective methodology — albeit with a false positive rate that is substantial and is likely to be higher than expected by many jurors based on longstanding claims about the infallibility of fingerprint analysis. The false-positive rate could be as high as 1 error in 306 cases based on the FBI study and 1 error in 18 cases based on a study by another crime laboratory. In reporting results of latent-fingerprint examination, it is important to state the false-positive rates based on properly designed validation studies.[103]

The PCAST insists that without insight into error, generated through well-designed validation studies, decision-makers are deprived of the means to evaluate the probative value of (or assign weight to) the examiner’s opinion.[104]

2 Responses to the Cross-Examination in JP

When we turn to JP, the contrast between authoritative scientific advice and the responses provided during cross-examination is stark. Although multiple reviews by scientific organisations have concluded that claims about infallibility and a zero error rate are ‘clearly ... unrealistic’[105] and ‘not scien-tifically plausible’,[106] the witness does not accept the possibility of error. Rather, he reiterates the discredited contention that the ACE-V procedure, properly applied, is error-free:

Defence Counsel: Do you agree that there are a number of potential sources of error associated with your ACE-V technique of fingerprint examination and identification?

Fingerprint Examiner: If the ACE-V methodology is done correct I don’t agree that there’s potentially error rates there.[107]

When pressed, the examiner asserted that he does not make mistakes — even if, on rare occasions, others might:

Defence Counsel: Is it possible that you have made a mistake or mistakes in your examination of fingerprint impressions in this case?

Fingerprint Examiner: No I haven’t.

Defence Counsel: Is it possible that you’re wrong about the accused being the source of the fingerprint impression in W3?

Fingerprint Examiner: No I’m not.

Defence Counsel: In every case in which you’ve identified a latent print to a known print have you been a hundred per cent certain?

Fingerprint Examiner: Yes I have.

Defence Counsel: You’ve never had any doubt?

Fingerprint Examiner: Never.[108]

And:

Defence Counsel: You agree don’t you that fingerprint examination and identification is subject to human error?

Fingerprint Examiner: There’s always an element of human error in anything we do.

Defence Counsel: But you say because of the method that you use that you’re always right when you make an identification is that what you say?

Fingerprint Examiner: Me personally yes.

Defence Counsel: So you would say that the ACE-V method is infallible is that what you say?

Fingerprint Examiner: In the correct — used in the correct method and way and by myself yes.[109]

The examiner is unwilling to accept that, even though ACE-V is an inherently subjective procedure and fundamentally dependent on human interpretation, it is vulnerable to error. His answers rely heavily on personal opinion and claimed abilities.

Defence counsel later endeavoured to question the witness directly on the NRC’s findings:

Defence Counsel: The [NRC] report also concludes in relation to fingerprint comparisons that,

Errors can occur with any judgment based method especially when the factors that lead to the ultimate judgment are not documented, some in the latent print community argue that the method itself if followed correctly ie: by well trained examiners properly using the method has a zero error rate. Clearly this assertion is unrealistic and moreover it does not lead to a process of method improvement. The method and the performance of those who use it are inextricably linked and both involve multiple sources of error, for example errors in executing the processed steps as well as errors in human judgment.

Are you aware of that part of the report?

Fingerprint Examiner: No I’m not.

Defence Counsel: Do you dispute that conclusion of the United States National Academy of Science?

Fingerprint Examiner: Parts of it yes I do.

Defence Counsel: Which parts?

Fingerprint Examiner: The parts that say by a well trained examiner properly using the method has a zero error rate.

Defence Counsel: You agree there is an error rate do you?

Fingerprint Examiner: Not if the ACE-V method is used correctly.

Defence Counsel: So you say that the zero error rate [sic] if a well trained examiner uses ACE-V method?

Fingerprint Examiner: Is zero yes that’s correct.

Defence Counsel: So you refuse to acknowledge that it’s possible that you made a mistake in this case?

Fingerprint Examiner: I did not make a mistake in this case that’s correct.

Defence Counsel: I tender the extract of the report your Honour.

EXTRACT OF REPORT TENDERED. OBJECTION.[110]

The witness’s response prevented the defence from tendering and relying upon extracts from these authoritative reports as evidence. The prosecutor’s objection, premised on the impropriety of tendering a report not recognised as authoritative, insulates the fingerprint examiner from criticism and deprives the Court of access to mainstream scientific knowledge.[111]

The rejection of the possibility of individual error is curious, because the examiner appears to eventually concede that fingerprint examination has an error rate, even though we are not always able to determine when an error has been made:

Defence Counsel: Do you agree it’s not possible to calculate the error rate for latent fingerprint identification because the truth is not known?

Fingerprint Examiner: If you put it that way yes.[112]

This concession seems to imply that ACE-V might not be an infallible method.

When prompted, the fingerprint examiner also acknowledged the existence of ‘documented’ errors in his field:

Defence Counsel: In practise we do not know how often examiners say that two fingerprints match when they actually come from two different people, do you agree with that?

Fingerprint Examiner: Unless it’s documented that’s correct.

...

Defence Counsel: And so in practise in the latent fingerprint field, do you agree that we don’t know how often examiners say that two fingerprints match when they actually come from two different people?

Fingerprint Examiner: Well it all comes down to documentation, if it’s documented we’ll know, if it’s not documented we don’t know.[113]

Within the broader context of the cross-examination, ‘documentation’ seems to refer to high profile false identifications, such as those relating to Mayfield and McKie.[114] But the possibility of error (as disagreement) also emerges when the witness is pushed about disagreement within NSW Police:

Defence Counsel: In your experience how often does a second or subsequent expert in the crime scene section or the FSG [Forensic Services Group] come to a different conclusion to an earlier examiner?

Fingerprint Examiner: It is rare but it does happen when an erroneous identification has been made.[115]

And:

Defence Counsel: Have you ever disagreed with another fingerprint examiner about whether two fingerprint impressions are from the same source or not?

Fingerprint Examiner: No I haven’t.

Defence Counsel: Never?

Fingerprint Examiner: Only when — on a working basis no but I have been asked to review a person who has made an erroneous identification.[116]

Personal experience with ‘rare’ cases of erroneous identification does not alter the examiner’s testimony or confidence in his own performance. His testimony might be taken to suggest that NSW Police’s internal verification processes are robust and that all, or most, errors are identified. However, it provides no basis to assess how often errors occur, including how often both evaluation and verification are mistaken.

On the issue of police errors, we should not overlook the fact that resol-ving the question of whether an identification is ‘erroneous’ is based on undisclosed internal police processes. In the absence of ground truth, disagreement is resolved through private discussion and consensus, or perhaps via ex cathedra pronouncements handed down by examiners with more experience or seniority. In addition, we have no independent way of knowing this particular witness’s performance or error rate across his many years of service.[117] In the absence of a calculated error rate, his assertions of error free performance are merely ipse dixit. Moreover, we have no evidence that this particular examiner is better or worse than other latent fingerprint examiners, including those with considerably less experience and those he characterises as more error-prone.[118] The studies reviewed by the PCAST provide the only reliable evidence on error rates currently available. The inst-itutions responsible for producing latent fingerprint evidence do not appear to apply systematic, transparent and effective methods for detecting, docu-menting, resolving or disclosing disagreement and error.[119]

D Expression: The Meaning of Match Decisions

1 Authoritative Scientific Findings and Recommendations

Historically, latent fingerprint examiners, and many other forensic scientists, have exaggerated the strength of their methods, abilities and conclusions.

In Part V(C)(1) we saw how the PCAST recommends incorporating an indication of error in the reporting of results.[120] Prior to these studies, the NRC Report, the NIST Report and the SFI Report had all recommended that latent fingerprint examiners should moderate the expression of results, because the leap from a conclusion that two areas of friction ridge ‘match’, to positive identification of a specific individual, is not supported by the available research.

Consider the following findings and recommendations:

At present, fingerprint examiners typically testify in the language of absolute certainty. ... Given the general lack of validity testing for fingerprinting; the relative dearth of difficult proficiency tests; the lack of a statistically valid model of fingerprinting; and the lack of validated standards for declaring a match, such claims of absolute, certain confidence in identification are unjustified ... Therefore ... fingerprint identification experts should exhibit a greater degree of epistemological humility. Claims of ‘absolute’ and ‘positive’ identification should be replaced by more modest claims about the meaning and significance of a ‘match’.[121]

And:

[A] fingerprint identification was traditionally considered an ‘individ-ualization,’ meaning that the latent print was considered identified to one finger of a specific individual as opposed to every other potential source in the universe. However, the recent attention focused on this issue reveals that this definition needlessly claims too much, is not adequately established by fundamental research, and is impossible to validate solely on the basis of experience. ... [E]xaminers should not claim to be able to exclude every other finger in the world as a potential source. Rather, an identification decision suggests a substantial enough similarity that the examiner believes that the two impressions originated from a common source.[122]

The Scottish Fingerprint Inquiry recommended:

Examiners should discontinue reporting conclusions on identification or exclusion with a claim to 100% certainty or on any other basis suggesting that fingerprint evidence is infallible.[123]

More recently, the American Association for the Advancement of Science (the ‘AAAS’) concluded:

Latent print examiners traditionally claimed to be able to ‘identify’ the source of a latent print with 100% accuracy. These claims were clearly overstated and are now widely recognized as indefensible.[124]





2 Responses to the Cross-Examination in JP

Working against the clear consensus that emerges from authoritative scientific research and systematic reviews from the US and UK, the examiner identified JP as the source of the crime scene print:

Defence Counsel: And do I understand correctly that in your opinion the crime scene fingerprint matches the accused’s fingerprint to the exclusion of all other people?

Fingerprint Examiner: Yes that is correct.

Defence Counsel: So is it your opinion that the accused is the only possible source for the fingerprint impression recovered from the crime scene at W3?

Fingerprint Examiner: Yes.

Defence Counsel: What’s your level of confidence in relation to that opinion?

Fingerprint Examiner: 100 per cent.

Defence Counsel: You’re a hundred per cent certain about that conclusion?

Fingerprint Examiner: Yes I am.[125]

Evidently oblivious to scientific consensus, this examiner expresses his conclusion in a manner that has been expressly disapproved in every authoritative scientific review of latent print examination.[126] His testimony misrepresents the strength of latent fingerprint evidence and the magnitude

of error.

The NIST Report was prepared by a working group composed of eminent scientists, lawyers and some of the most senior latent fingerprint examiners in the US. When presented with its recommendation, the examiner steadfastly adheres to his opinion:

Defence Counsel: One of the recommendations by the United States National Institute of Standards and Technology was this, ‘Because empirical evidence and statistical reasoning do not support a source attribution to the exclusion of all other individuals in the world latent print examiners should not report or testify directly or by implication to a source attribution to the exclusion of all others in the world.’ What do you say about that recommendation?

Fingerprint Examiner: Well it’s someone’s opinion in America.

Defence Counsel: Do you agree that your report and evidence today are not consistent with that recommendation?

Fingerprint Examiner: If you are guided [by] that recommendation no they’re not consistent.

Defence Counsel: In light of this recommendation do you stand by the opinion that you’ve given here that the crime scene fingerprint W3 can be attributed to JP exclusively?

Fingerprint Examiner: Yes I can.[127]

This exchange is revealing. This examiner is unfamiliar with the NIST Report, and other scientific reports, on latent fingerprint analysis. And yet, in the context of a criminal proceeding, and with a fundamental duty to assist

the Court impartially, he dismisses this authoritative recommendation

as ‘someone’s opinion in America’.[128] Unwilling to make any concession and, in effect, acting against the advice of peak scientific and technical organisations, the examiner (in his capacity as an expert and a representative of the state) declines an opportunity to reconsider his position and commitments or even to engage more deeply with the literature on which he is being cross-examined.[129]

E Bias, Blinding and Non-Blind Verification

1 Authoritative Scientific Findings and Recommendations

Historically, risks from human factors have been overlooked or discounted by forensic practitioners and courts.[130] The NRC Report placed unprecedented emphasis on risks from cognitive bias:

Two very important questions should underlie the law’s admission of and reliance upon forensic evidence in criminal trials: (1) the extent to which a particular forensic discipline is founded on a reliable scientific methodology ... and (2) the extent to which practitioners in a particular forensic discipline rely on human interpretation that could be tainted by error, the threat of bias, or the absence of sound operational procedures and robust performance standards.[131]

Cognitive bias was also addressed by the NIST and the PCAST:

Cognitive bias refers to ways in which human perceptions and judgments can be shaped by factors other than those relevant to the decision at hand. It includes ‘contextual bias,’ where individuals are influenced by irrelevant background information; ‘confirmation bias,’ where individuals interpret information, or look for new evidence, in a way that conforms to their pre-existing beliefs or assumptions; and ‘avoidance of cognitive dissonance,’ where individuals are reluctant to accept new information that is inconsistent with their tentative conclusion. The biomedical science community, for example, goes to great lengths to minimize cognitive bias by employing strict protocols, such as double-blinding in clinical trials.[132]

Concern with cognitive bias is integral to a full understanding of the strength of expert evidence. Cognitive bias may inadvertently affect the accuracy of human judgment even where experts are formally qualified, experienced and aware of the dangers. These risks have led biomedical researchers and physicists to routinely blind themselves to information that might influence

their practices or analyses.[133] Vulnerability to cognitive bias is not the result of dishonesty or inadequate training and experience:

To recognize that latent print examiners are potentially subject to bias is not to single them out but rather to suggest that they are not exempt from those cognitive biases that all interpreters of data and information face.[134]

On the basis of these insights, the NIST Report offered advice and recommendations on how to manage information and work practices to reduce the threat posed to forensic science evidence. Among the NIST’s recommendations, the following was prominent:

Procedures should be implemented to protect examiners from exposure to extraneous (domain-irrelevant) information in a case.[135]

Similar recommendations were made by the PCAST,[136] the AAAS[137] and the US National Commission on Forensic Science (established in response to the NRC Report).[138] The US National Commission recommended that:

1 FSSPs [forensic science service providers] should rely solely on task-relevant information when performing forensic analyses.

2 The standards and guidelines for forensic practice being developed by the Organization of Scientific Area Committees (OSAC) should specify what types of information are task-relevant and task-irrelevant for common forensic tasks.

3 Forensic laboratories should take appropriate steps to avoid exposing analysts to task-irrelevant information through the use of context management procedures detailed in written policies and protocols.[139]

Because the risk of contextual and other unconscious biases ‘cannot be dismissed’, the NIST recommended documentation and disclosure in the absence of blinding.[140] Accordingly, expert reports ‘should reveal the context of the examination by describing or referring the reader to the information about the case that an examiner received’.[141]

2 Responses to the Cross-Examination in JP

The fingerprint examiner, and those verifying his decision, were all

unnecessarily exposed to domain-irrelevant information (ie task-irrelevant information). Notwithstanding a lack of familiarity with relevant research

and revised procedures, the fingerprint examiner downplayed well-

documented risks:

Defence Counsel: Do you claim to be immune to any form of bias in your work as a fingerprint examiner?

Fingerprint Examiner: In my own mind yes.

Defence Counsel: Are you familiar with the term cognitive bias?

Fingerprint Examiner: I’ve heard of the term but I’m not quite familiar with it exactly what it means no.

Defence Counsel: Are you familiar with the concept of contextual bias?

Fingerprint Examiner: Again I’ve heard that term but I’m not familiar with the actual wording of it.

Defence Counsel: What’s your understanding of the term contextual bias?

Fingerprint Examiner: Well not exactly contextual bias but bias is that something is put in front of you and it leads you to a determination.

Defence Counsel: Are you familiar with the term ‘domain irrelevant information’?

...

Fingerprint Examiner: No I’ve never heard of that term.[142]

And:

Defence Counsel: And are you saying that there was no role to play for confirmation bias when you came to identify W3 with JP’s left thumb in circumstances where you had all that information?

Fingerprint Examiner: Again I say that that information from those people is purely a hypotheses [sic] and it’s up to me to either confirm or deny that hypotheses [sic].

Defence Counsel: So okay you wouldn’t agree with this proposition, that there were a number of pieces of information that you had when you undertook your examination in this case of W3 and JP’s left thumb impression, that mean you were susceptible to confirmation bias?

Fingerprint Examiner: No I wouldn’t agree with that.

Defence Counsel: Are you aware of the scientific literature in relation to fingerprint examination and identification being vulnerable to a number of forms of bias?

Fingerprint Examiner: I am aware that there is bias involved or can be involved.

Defence Counsel: Including when ACE-V is used?

Fingerprint Examiner: If ACE-V is used correctly bias shouldn’t be taken into account at all.[143]

Here, the witness conveys some awareness that ‘bias ... can be involved’.[144] Nevertheless, he rejects the possibility of bias and error in relation to his own performance. For example, he testifies that because a match by another examiner is treated as a ‘hypothesis’, it does not pose a risk of biasing his judgment when reviewing the decision. This is a good example of a seemingly plausible claim that is in fact inconsistent with scientific research and mainstream scientific advice.[145] The examiner, whether he realises it or not, is susceptible to both contextual bias (eg suggestion) and confirmation bias, when he conducts his examination with prior knowledge of the judgment of another examiner. No scientific research suggests that the risk posed by bias can be avoided by treating potentially biasing information or processes as a ‘hypothesis’ — particularly in the absence of any procedures designed to seek and document both disconfirmatory and confirmatory information.[146] Biomedical researchers and physicists do not employ ‘hypotheses’ to circumvent the risks posed by cognitive bias. Rather, they employ

procedures that eliminate the dangers by blinding themselves to domain-irrelevant information.

When questioned about two notorious studies of cognitive bias — by Dror and his colleagues[147] — the examiner indicates some awareness but cannot offer any assistance.[148] These studies showed that experienced fingerprint examiners reversed their opinion about whether two prints matched when — unbeknown to them — they were asked to compare the same prints in different circumstances.[149] On the second occasion they were exposed to suggestive information about the source of the prints. When asked about these studies:

Defence Counsel: That demonstrate even experienced latent fingerprint examiners can change their mind?

Fingerprint Examiner: Well I couldn’t say exactly what ones but that comes down to the personal opinion of that person.[150]

It is unclear what the examiner means when he asserts that these studies are a matter of ‘personal opinion’. Does he mean the personal opinion of the authors of the study, or those of the examiners who participated? In either sense, his response reveals a lack of familiarity with this research and its implications for practice. These studies confirm that experienced fingerprint examiners are vulnerable to suggestion and confirmation bias, to the point where they can be induced to contradict their own previous match decisions.

Cognitive bias may influence interpretations even when the examiner is experienced and honest, and even when they are aware of the dangers. It can be difficult to effectively cross-examine a witness on the subject: How does one cross-examine a witness about subtle influences on cognition that may operate below the level of consciousness? Even though it might be possible to question a witness about generic dangers or procedures that might have been implemented to avoid the risks, the issue cannot be meaningfully explored where, as in JP, the witness is not sufficiently conversant with the concept of cognitive bias.[151]

F Uniqueness

1 Authoritative Scientific Findings and Recommendations

Claims that objects are unique, or that they leave unique traces, are ubiquitous among comparison forensic science communities. Firearms are said to leave a unique mark on a bullet casing or projectile; teeth are said to leave unique bite marks; and, faces, voices, gait, handwriting, tools and fingerprints have each been said to be unique.[152] Uniqueness has a special salience because, in theory, if an object or a trace is unique and can be unequivocally matched to a particular source (eg a specific gun or a given suspect), then questions about its origin or identity are resolved. However, claims to uniqueness tremend-ously simplify the complexities of comparison and the implications of similarities between a trace and its suspected source. Consider the following assessment by the PCAST:

[U]niqueness studies miss the fundamental point. The issue is not whether objects or features differ; they surely do if one looks at a fine enough level. The issue is how well and under what circumstances examiners applying a given metrological method can reliably detect relevant differences in features to reliably identify whether they share a common source. ...

Moreover, it is not necessary for features to be unique in order for them to be useful in narrowing down the source of a feature. Rather, it is essential that there be empirical evidence about how often a method incorrectly attributes the source of a feature.[153]

And, the assessment by the AAAS:

The scientific literature does not, however, provide an adequate basis for assessing the rarity of any particular feature, or set of features, that might be found in a fingerprint. Examiners may well be able to exclude the preponderance of the human population as possible sources of a latent print, but there is no scientific basis for estimating the number of people who could not be excluded and there are no scientific criteria for determining when the pool of possible sources is limited to a single person.[154]

2 Responses to the Cross-Examination in JP

The proposition that fingerprints are unique is commonly presented and widely understood as unshakeable fact.[155] It is tightly coupled to how finger-print identification was initially conceptualised and explained. We can ob-serve the commitment to uniqueness and the work it does for categorical identification in the following interactions:

Defence Counsel: Officer is your opinion about the match between [JP’s] impression and the crime scene impression based on an assumption that no human being throughout history has identical fingerprints?

Fingerprint Examiner: Yes that is correct.

Defence Counsel: That’s commonly referred to as the uniqueness of human fingerprints is that right?

Fingerprint Examiner: That is correct yes.

Defence Counsel: The assumption that all human beings have unique fingerprints is a premise relating to the friction ridge skin is that right?

Fingerprint Examiner: Yes that’s correct.[156]

...

Defence Counsel: So do you accept that it’s impossible to say that every human being has a unique set of fingerprints?

Fingerprint Examiner: No I accept that they are all unique.

Defence Counsel: Do you accept that it’s impossible to say that each digit or finger belonging to all the human beings throughout history are unique as against every other digit?

Fingerprint Examiner: No I don’t accept that.[157]

Reliance upon the uniqueness of fingerprints operates as something of a smokescreen — it is likely to mislead or confuse those who are not familiar with scientific approaches to forensic feature comparison procedures. Ironically, it has had this effect on many among the latent fingerprint comm-unity, and we can see its effects in these and other exchanges.

The problems spill over into the frequency of fingerprint features (or minutiae). The examiner acknowledged that he does not know about the frequency and interrelatedness of features:

Defence Counsel: Do you agree that you cannot give evidence about the actual frequency of particular characteristics or combinations of characteristics in fingerprints in the human population?

Fingerprint Examiner: Yes.

Defence Counsel: For example you can’t give evidence about the frequency of left loops or double loops on fingers generally?

Fingerprint Examiner: Not the frequency no.

Defence Counsel: I may have all ready [sic] asked a question about this in relation to analysis but in relation to the evaluation phase again it’s based on the subjective view of each individual examiner is that right?

Fingerprint Examiner: Yes that is correct.[158]

Given this lack of baseline information, positive identification represents a leap of faith from a match decision.[159]

Significantly, the examiner acknowledged differences between the latent print and JP’s known print. These apparent differences were characterised

as explicable:[160]

Defence Counsel: How many differences did you detect between the impressions?

Fingerprint Examiner: Nil.

Defence Counsel: Are you saying that the two impressions were absolutely identical?

Fingerprint Examiner: No I’m not, what I’m saying is that there was nil unexplainable differences.

...

Defence Counsel: So do I understand your evidence correctly that you did discern some differences between the two impressions but all of the differences that you discerned you explained away?

Fingerprint Examiner: Yes that is correct.

Defence Counsel: Is it possible that there was a difference between the impressions that you were unable to detect?

Fingerprint Examiner: Between the two impressions?

Defence Counsel: Yes?

Fingerprint Examiner: Unable to detect you said?

Defence Counsel: Unable to observe or detect, unable to perceive?

Fingerprint Examiner: Well I did not perceive any unexplainable differences.[161]

According to the examiner, ‘apparent’ differences are created by the circum-stances of deposition and collection, rather than differences between the actual finger(s) responsible for the two prints. These observable differences were not documented and are resolved entirely in the examiner’s head. The decision as to whether a difference is real or explicable is a subjective inter-pretation with considerable import. There is nothing in ACE-V that addresses this issue and no standards around whether differences should be understood as real or artificial.[162]

The ideological (or metaphysical) commitment to uniqueness does not address or overcome the subjective nature of the interpretation, or the estab-lished fact that some fingerprints from different persons appear to be very similar and have been mistakenly matched.[163]

G Training, Continuing Education and Familiarity with Pertinent

Scientific Research

The previous extracts suggest that the fingerprint examiner has not engaged with mainstream scientific research, criticisms and recommendations. Accor-dingly, it is useful to consider his testimony about efforts to remain abreast of pertinent developments in the field and what this testimony might reveal about NSW Police training and continuing education:

Defence Counsel: Do you keep abreast of the current available scientific research in relation to fingerprint identification?

Fingerprint Examiner: I read available documentation that I have at Dubbo Crime Scene when time permits.[164]

...

Defence Counsel: Do you stay up to date with the science related to fingerprint examination identification?

Fingerprint Examiner: I read documents that are sent to me by the training area periodically.[165]

...

Defence Counsel: Is it your evidence that in relation to your expertise and staying up to date with the fingerprint field you basically rely on whatever the New South Wales Police training section sends to you?

Fingerprint Examiner: Updated versions and methodology yes.

Defence Counsel: You don’t do anything of your own initiative to remain up to date in the field, is that right?

Fingerprint Examiner: Unfortunately time does not permit within my area.[166]

The examiner’s responses indicate that he relies upon the NSW Police training section to identify the information he needs in order to perform his role, and that his workload inhibits continuing education. Cross-examination revealed that neither he, nor the NSW Police training section, appear to consider familiarity with authoritative scientific research as being important to his role as an expert. Consider the following:

Defence Counsel: You’re of course aware aren’t you sergeant of the United States National Academy of Science Report, ‘Strengthening the Forensic Sciences in the United States: A Path Forward’, published in 2009?

Fingerprint Examiner: No I haven’t read that document.

Defence Counsel: Not aware of it at all?

Fingerprint Examiner: No.[167]

...

Defence Counsel: Following the 2009 report of the National Academy of Science there was a review carried out by the United States National Institute of Standards and Technology in relation to latent fingerprint evidence, are you aware of that?

Fingerprint Examiner: No I’m not.

Defence Counsel: 2012 they released their report ‘Latent Print Examination and Human Factors: Improving the Practise Through A Systems Approach’ and they made a number of recommendations, are you aware of that?

Fingerprint Examiner: No I’m not.

Defence Counsel: You’re not aware of major international reporting and recommendations in your field of expertise, is that what you’re saying?

Fingerprint Examiner: I don’t go searching for that information if the — if our training area feels that it is deemed necessary to know they send the appropriate literature to read.[168]

These interactions are revealing. A significant proportion of the very modest Expert Certificate prepared for the proceedings in JP refers to the examiner’s training and experience and the various short courses he had attended. While this Expert Certificate asserts that he is an expert in the field of latent fingerprint examination and refers to his ‘specialised knowledge’,[169] the cross-examination introduces uncertainty about its scope.[170]

This latent fingerprint examiner, the senior examiner in the region, is unaware of perhaps the most important developments in the field over its 100-year history. He is unaware that the ACE-V procedure, which he describes as infallible, was not formally tested until after 2009. He does not seem to be familiar with the results of these studies and the indicative error rates they generated. This is important. The reports and studies are not merely some ‘journal’ article or ‘someone’s opinion in America’.[171] Rather, they are vitally important scientific reviews produced by multidisciplinary teams of independent scientists and focus on the very methods employed by the latent fingerprint examiners of New South Wales (‘NSW’). These reports (and the research and methods they draw upon) challenge both the procedures and the conclusions produced by NSW latent fingerprint examiners. They challenge the scope of the witness’s expertise and are incompatible with his opinion

and testimony.

When some of these inconsistencies are brought to the attention of the witness, he does not defer or take the opportunity to reflect upon his training and commitments:

Defence Counsel: Do you dispute that conclusion of the United States National Academy of Science?

Fingerprint Examiner: On the grounds of fingerprints I would yes.

...

Defence Counsel: Do you dispute that conclusion of the United States National Academy of Science?

Fingerprint Examiner: Under the current methodology of New South Wales Police yes I do.[172]

In JP, the state called a fingerprint examiner who was neither scientifically trained nor conversant with scientific research, scientific advice, and metho-dological and other problems with his procedures. He was incapable of providing a framework through which to make sense of his opinion — an opinion on the ultimate issue in the proceedings.

H The Code of Conduct for Expert Witnesses

The fingerprint examiner signed an Expert Certificate stating that he was familiar with the Expert Witness Code of Conduct (the ‘Code’)[173] and acknow-ledging that his primary obligation was to impartially assist the Court.[174] The preceding discussion might help the reader to appreciate the limitations of the Code (and similar normative frameworks) for witnesses who lack fund-amental knowledge of their ‘field’. The exchange below suggests that, not-withstanding the Code, this latent fingerprint examiner did not consider it his duty to disclose uncertainty, concerns, or criticism of his procedure or

the field:

Defence Counsel: You understand that you have an overriding duty to assist the court and to do that impartially?

Fingerprint Examiner: Yes I do.

Defence Counsel: And part of acting impartially as an expert witness involves disclosing the existence of uncertainty, concern or criticism in your particular field, do you agree with that?

Fingerprint Examiner: I would — if I had that uncertainty within the case that I was looking at I would definitely disclose that.

Defence Counsel: You don’t consider your role as an expert in acting impartially and assisting the court to inform the court about a general level of uncertainty, concern or criticism in your field. Is that what you’re saying?

Fingerprint Examiner: I don’t think it would be my duty as a witness in a particular case to do that.[175]

The Code stipulates that an expert should disclose ‘any qualification ... without which the report is or may be incomplete or inaccurate’.[176] The previous exchange suggests that this examiner, even if he was familiar with the scientific research, limitations and uncertainties regarding latent print examination, would not have disclosed them.[177] It suggests that the only qualifications the witness would disclose are personal (or subjective) doubts in relation to a particular comparison or conclusion. This interpretation of the Code — along with belief in the infallibility of the ACE-V method and the naive contention that he has never made a mistake — is used to insulate the examiner from having to refer to any research or criticisms, or to make disclosures. As he testified in passages reproduced earlier, whenever the examiner reports a match he experiences no doubts, and so there will never be a need to disclose any limitations — however critical or destabilising.

The previous exchange, and the testimony as a whole, captures the super-ficial manner in which the Code tends to inform the practice and disclosure made by the state’s forensic scientist. If a witness does not know about, understand or accept mainstream scientific research, criticisms and recomm-endations, then, on this witness’s interpretation of the Code, there is no need to refer to them.

This section begs the question of what it means for a witness to be, or hold themselves out to be, impartial (or even expert) if they are not conversant with, or do not disclose, authoritative, rigorously conducted research directly related to their procedures and abilities.[178] The rules of evidence accord expert witnesses a special status on the premise that they offer knowledge-based and impartial assistance to the court. In JP, answers provided during cross-examination are expressed and defended in ways that systematically advance the state’s accusation while departing from what is actually known.

VI Injudicious Credulity: Legal Evaluation of the Latent Fingerprint ‘Evidence’

In this section, we consider the Magistrate’s reasoning before moving on to discuss issues flowing from this reasoning and the cross-examination. Let’s begin with the actual decision (upheld on appeal).[179] Two important observations frame our analysis of this decision. At trial, JP’s case was heard by a magistrate. That is, a judicial officer who does not have the time and res-ources available to judges more senior in the court hierarchy. Secondly, the scientific materials raised by the defence were not in evidence. With these constraints in mind, what follows are the formal findings on the evidence presented at trial:

I accept that fingerprints may be unreliable, may be affected by bias. I accept that two experts can have a disagreement or a different opinion on a set of fingerprints. I accept historically errors appear to have been made.

In this matter I have oral and written evidence from [the fingerprint examiner]. His evidence was unshaken on his view as to the matching of the thumbprint of [JP]. In my view I disagree with the submissions in this matter, he has given sufficient evidence in these proceedings as to how he reached that determination. As an expert his expertise was not shaken, his opinion was not shaken. He is tasked, as he said, purely to compare W3 to [JP’s] prints. There is clearly in terms of the procedures involved, checks and balances in place. He acknowledged he is aware of case studies where potential impacts and bias of proceedings have occurred. His view as the expert in the field or presented as the expert in this matter is that where the appropriate procedures have taken place, [it] is unlikely to have those errors occur. He also conceded that he had not read a lot of the literature referred to by [counsel for the defence] in the cross-examination. Again he maintained his view that if protocol was followed properly it should not involve bias or incorrect assessment.

The difficulty of course with a lot of material that was cross-examined on is there is no method, no chance to actually test the validity of those arguments. I note that a lot of that material, there is a report there from 2008 or 2009 but I do not have the actual name in front of me, and a follow up report in 2012 provided to judicial bodies of the United States. There is no evidence of any action taken on those views by those researchers or otherwise. The report from Scotland does not take it any further than his opinions being on the balance of probabilities that would assist further inquiries. No great depth as to what the actual error was and how that could potentially relate to the matter in this matter.

I have no evidence before this Court of the method used in this instance by [the fingerprint examiner] ... (not transcribable)... helping assist in his determination was tainted by the bias or other incorrect assessment by not following the protocols. I have no expert evidence on the defence showing in this particular matter that the thumbprint is not or could not be the accused’s. I say that of course there remains at all times the prosecutions responsibility to prove the matter beyond reasonable doubt. It was suggested that [the fingerprint examiner] was contradictory or failed to make proper concessions, I actually find to the contrary. He answered appropriately in all circumstances especially where the questions were extremely open-ended and hypothetical. He did not attempt in any way to make his evidence or his position any greater than what it should in terms of the protocols that were involved.

It is put to the Court that the judicial officer could not assess the prints from the photos that are presented. It is not this Court’s role to be the expert, that is why the police call an expert to give their opinion and why, if required the defence call an expert as to why it is not, it is not the Court’s role to look at the various rigors and make a determination and even [the fingerprint examiner] on his own evidence indicated he needed specialist equipment to be able to identify it.

The evidence by [the fingerprint examiner] in giving his opinion in determination has not been proved forensically challenged in this matter. There is no Court decisions making such material unacceptable. ... At best I have nothing else binding before me that would exclude the evidence of [the fingerprint examiner]. I can only scrutinise it on the material before me specific to this case. I accept [the fingerprint examiner’s] evidence in that regard.

...

I said anything is possible but is it probable? It is [improbable] that a person with the same thumbprint is in the Dubbo area and has entered the subject premises. Taking into account historically — and I agree with the defence submissions about the lack of a worldwide data base for eternity — taking into account historically there is no record of two persons with the same print and there may be, it may come up in the future but at this stage I can only take judicial notice of what is out there at the moment. That [there were] two persons with the same print in existence at the same time in Dubbo is highly improbable and that another person would be at those premises. I have no other evidence as to how the thumbprint came to be on that jewellery case in the circumstances that it was examined and identified. I need to determine beyond reasonable doubt, not a hundred percent and whilst hypothetically some doubt has arisen in my view there is no reasonable doubt in this matter.[180]

His Honour found the offence proven.[181] The reasons appear to accept that identification by fingerprint may be ‘unreliable’, ‘affected by bias’ and that ‘errors appear to have been made’ in other cases.[182] Further, they record that the witness ‘acknowledged ... case studies where potential impacts and bias ... have occurred’.[183] The Magistrate accepts the fact, though perhaps not the implications, of there being no ‘worldwide data base for eternity’.[184] Formal recognition of these risks and uncertainties is tempered by the availability of the Expert Certificate (or report) and the positive responses obtained during cross-examination and re-examination.

In evaluating the latent fingerprint evidence, the Magistrate seems to assume that procedural anomalies and limitations — such as lack of standards, non-disclosure of an indicative error rate, and exposure to domain-irrelevant information — exert no influence on the examiner’s conclusion.

For the Magistrate, the witness’s ‘expertise was not shaken’ and ‘his opinion

was not shaken’.[185] Remarkably, his ‘opinion’ was reported as ‘not ... forensically challenged’.[186] He was recognised as an expert and his various opinions were accepted even though ‘he had not read a lot of the literature’.[187] The examiner’s bare description of his procedure — visually comparing two prints on a computer screen — was characterised as ‘sufficient’ explanation of the reasoning to support categorical identification.[188] The Magistrate accepts that ‘clearly’ there were ‘checks and balances in place’, although he does not identify them or analyse their utility.[189] The Magistrate accepts, indeed repeats, the ‘view’ that where ‘the appropriate procedures have taken place’ it is ‘unlikely’ that errors will occur.[190] This evaluation relies on the claims of a latent fingerprint examiner and upon past legal practice but is insensitive to, and inconsistent with, scientific knowledge.

The reasons allude to the authoritative scientific materials — the various reports that are not named in the Magistrate’s decision (or the appellate judgment). The Magistrate, who was in an undoubtedly invidious position, indicates that he was not able to ‘test the validity’ of the research and conclusions — which he characterises as ‘arguments’.[191] Further, he holds that there was ‘no evidence of any action taken on those views’.[192] There was no evidence led about the FBI having changed its practices (by adopting ‘linear ACE-V’) in response to the NRC Report and another inquiry, or about the US Federal Government establishing the National Commission on Forensic Science and a series of technical committees under the oversight of the NIST to undertake research and advise forensic science providers. The problem, of course, is that because the witness did not accept the authority of the NRC Report and the NIST Report, or their findings and recommendations,

there was, in effect, no evidence, apart from the examiner’s opinions, before

the Court.

As for the examiner, in response to the defence submission that he was ‘contradictory or failed to make proper concessions’, the Magistrate found ‘to the contrary’.[193] He was said to have answered ‘appropriately in all circumstances’ to questions derisively characterised as ‘extremely open-ended’.[194] Questions directed at his method and, its limitations, risks and errors, are treated as theoretical — things that are ‘possible’ but not ‘probable’.[195] Attempts to introduce materials that question the procedure and the witness’s impression were treated as ‘hypothetical’.[196] However, the PCAST does not consider the concerns raised by the NRC (and the NIST) or the results of the scientific research it relies upon, to be theoretical, hypothetical or abstract issues somehow displaced from real world practice.[197] The fact that the research is general and not concerned with the contingencies of specific cases does not lead these scientists to dismiss the evidence of error as general or remote from practice.[198]

The Magistrate displays a traditional and exclusive interest in the opinion presented in the instant case. The reasons communicate an expectation that the defence must identify an actual error.[199] Again, this expectation is misguided, because scientific advice indicates that the examiner’s method is not known to eliminate errors. Furthermore, the NSW Police did not take any precaution against the risk of cognitive bias.[200] Risk of error is invidious because, in most cases, the examiner and court will not know if it materialises. This is why the reports recommend validation and precautionary blinding.[201] Apparently misunderstanding the nature of these dangers, the Magistrate seems to expect the defence to show that the ‘thumbprint is not or could not be the accused’s’.[202] The various concerns raised by the defence, but not accepted by the latent fingerprint examiner, are accordingly seen as inapposite or insignificant. In the absence of an identified error, the Magistrate accepts the latent fingerprint examiner’s opinion as proof of identity to the requisite standard.

Reverting to uniqueness, the Magistrate takes ‘judicial notice’ of the fact that ‘there is no record of two persons with the same print’.[203] The Magistrate is comforted by the fact that it is ‘improbable’ that two persons ‘with the same print’ would be in Dubbo.[204] The reasons illustrate the failure to grasp the true issue at stake in this case. The question is how often latent fingerprint examiners match prints from different persons and how often they fail to match prints made by the same person. The question is one about the likelihood that fingerprints from two different people could be confused or are so similar as to be capable of being confused (and how similar they would need to be for this to take place). Uniqueness neither addresses nor overcomes this fundamental issue.[205] Moreover, matching decisions may be biased by exposure to domain-irrelevant information; such as knowledge of the fact that the prints being compared both originated in Dubbo. A person who believes in uniqueness might be more likely to treat similarities in two prints from Dubbo as sufficient for identification than a person who does not know anything about the source of the prints they are asked to compare. What appears improbable (to the Magistrate) may contaminate the formation of the examiner’s opinion. This is how insidious cognitive bias can be.[206]

Despite acknowledging the presence of some hypothetical doubt, the Magistrate is satisfied that ‘there is no reasonable doubt in this matter’.[207] This decision was upheld on appeal, where the Court accepted that there was ‘no material’ — ie admissible evidence — to affect the weight of the examiner’s opinion or his credibility.[208] Moreover, according to the appellate court, the trial judge had the ‘distinct advantage’ of seeing the examiner’s demeanour:

While a number of criticisms were made of [the fingerprint examiner’s] evidence it was open to his Honour to conclude that there was no material to indicate that, to the extent the criticisms were sustained, they materially affected the weight to be attached to [the fingerprint examiner’s] opinion that the fingerprints were identical. Otherwise his Honour had the distinct advantage of being able to observe [the fingerprint examiner] give evidence and respond to criticism.[209]

This reasoning, and the dependence on admissible ‘evidence’, might be legally orthodox and considered reasonable by the appellate court and other lawyers and judges. However, we hope that readers will share our view that the insensitivity to scientific knowledge and, the disparities between scientific knowledge and the testimony, and reasoning, on display in this case

are disquieting.

VII Discussion: Error and Legal Ignorance

To what extent is it fair to judge the quality of responses provided by a latent fingerprint examiner in the Children’s Court in Dubbo against a standard set by pre-eminent research scientists and government technical organisations? While we accept that cross-examination can be a formidable experience, we consider that an expert’s duty to provide impartial assistance to the court requires experts, and their institutions, to familiarise themselves with authoritative research and recommendations about their procedures and abilities. The issues we raise in this article are not new and yet the gaps, between the testimony and scientific knowledge, are substantial. The NRC Report was published in 2009, the SFI Report in 2011, the NIST Report in 2012, and our own paper — ‘How to Cross-Examine Forensic Scientists: A Guide for Lawyers’, on which this challenge was partially based — was published in 2014.[210] The NSW Police were (and remain) on notice about these, and other, scientific reports. Their main response to these critical reports, prior to the case and appeal, seems to have been to supplement their training materials with ‘readings’.[211] Revealingly, the fingerprint examiner in JP testified that he was not aware of the scientific reports. His ignorance did not affect his certification and seniority, nor the accreditation of his bureau and the reports it routinely produces. Ignorance did not moderate his confidence or the expressed strength of his opinion. It did not affect judicial impressions of his credibility or the weight of his evidence, and did not render any of his evidence inadmissible — not even the parts based on his non-scientific impressions, beliefs and speculation.

The issues explored in this article are fundamental to practice, as well as the validity and scientific reliability of latent fingerprint evidence. It is not unreasonable to expect a forensic science practitioner, recognised as an expert witness, to remain abreast of scientific research on the very procedures they employ. The fact that the expert witness was incapable of addressing the questions being asked in cross-examination, combined with the Court’s inability to appreciate the significance of those questions and answers, would seem to be a fundamental problem. These are questions that the state should have been in a position to answer. It should have addressed many of them pre-emptively in the Expert Certificate (or report). Moreover, the state could have called other witnesses rather than rely on what were, in effect, dogmatic assertions given under the guise of expertise. The legitimacy of our criminal courts and investigative agencies depends on their ability to sensibly respond to scientific research and advice (as well as its absence).

We now turn to consider two issues with systemic implications. We start with the expectation that the defence might be expected to identify an error, and the implications of that expectation on the accusatorial trial and the burden of proof, before moving on to consider legal ignorance.

A Expecting the Defence to Identify an Actual Error

The Magistrate stated:

I have no expert evidence on the defence showing in this particular matter that the thumbprint is not or could not be the accused’s. I say that of course there remains at all times the prosecutions [sic] responsibility to prove the matter beyond reasonable doubt.[212]

There does, however, seem to be an expectation that the defendant might,

or could, have identified an error if one had been made. Yet, placing such a burden upon the defendant is inconsistent with orthodox accusatorial prin-ciples and practice.[213]

Those accused of criminal offences, including those who are factually innocent, will almost never be in a position to demonstrate positively that a forensic practitioner made an error, especially where opinions are based on subjective feature comparisons.[214] Errors will rarely be conspicuous on the documentary record. Identifying an error will usually require technical sophistication (and, in the context of the case, rigorous analysis). This will often be a necessary, though hardly sufficient, condition. The fact that collection, labelling, transportation, storage, processing, analysis, comparison, interpretation, verification and reporting occur remote in time and space from the defendant — and any expert assistance — merely accentuates the problem. Anaemic documentation, and the reluctant disclosure of protocols, procedures and notes, make it difficult, even for the technically proficient, to identify derogation, let alone improprieties and mistakes.[215] The defendant depends on the state to use procedures that do not unreasonably introduce, or increase, the risk of difficult-to-detect errors.[216]

The expectation that a defendant will identify an error, rather than req-uiring the state to eliminate the reasonable possibility of error, places the defendant at an evidentiary disadvantage and acute vulnerability in those cases where an error has been made. Difficulties are compounded where the procedures are subjective (ie interpretative). These take place in the exam-iner’s head. How then does the defendant demonstrate that an examiner is mistaken? How does the defendant ascertain if the subjective decision was influenced by cognitive factors that operate unconsciously? Practically, in the absence of information about the true perpetrator, how does an innocent defendant demonstrate that a difference deemed ‘explainable’ by an examiner (and explained away, if asked) is actually significant?

The Magistrate’s emphasis on ‘attacking’ the fingerprint evidence ‘directly’ and the need to identify an error (or expert disagreement) trivialises the serious methodological problems raised by the defendant in JP. It values the subjective opinions of forensic practitioners — including those who lack knowledge about their procedures — above mainstream scientific research, advice and insights related to those very procedures.[217] There was, as the cross-examination made clear, an error rate associated with the ACE-V procedure, and the procedure used in JP was performed in ways that have been shown to increase the likelihood of error. It was incumbent on the prosecution to eliminate the real possibility of error. Trial and appellate judges should explain the terms on which the criminal justice system is willing to engage with opinions and why, given the various problems raised (and the lack of other evidence in this case), reasonable doubt could be eliminated. Judicial reliance on an expert witness’s experience or confidence, or imputing a ‘failure’ to the defence, does not somehow repair procedures that have been demonstrated to have real frailties.

Frailties and uncertainties associated with the use of procedures are not abstractions. They inform how we should understand results obtained using those very procedures.[218] The value of opinions derived using ACE-V is informed by the results of formal evaluation. Their value should incorporate uncertainties, limitations and risks. To attribute a higher value than the results of formal evaluation can support is speculative.[219]

B Ignoring Knowledge

JP provides a clear example of why our current approach to authoritative scientific materials, including cross-examination on them, is unsatisfactory. It is unsatisfactory because it can deprive courts of the most reliable research available, in circumstances where a witness, recognised as an expert, is ignorant, obtuse or dishonest. In a context of shrinking resources and structural impediments to securing the services of a rebuttal witness, cross-examination has become the primary means of challenging expert opinion evidence in criminal trials. Institutional (and individual) insensitivity to mainstream scientific research threatens both the fairness of proceedings and, in some cases, including some guilty pleas, the rectitude of outcomes.

We have already recounted, in Part V(G), the fingerprint examiner’s reaction to the NRC Report and the NIST Report. He did not recognise their authority or accept their conclusions to the extent that they were inconsistent with ‘the current methodology of New South Wales Police’.[220] In addition, the witness was asked about scientific research, by Itiel E Dror and colleagues, included within the current NSW Police training materials.[221] Consider the Magistrate’s reaction to the witness’s limited familiarity with these materials:

Magistrate: He hasn’t acknowledged their scholarly orders. He’s acknowledged they’re part of the [NSW Police training] module. That’s all that he’s acknowledged at this stage, he hasn’t read them, he doesn’t know them. He hasn’t said he knows them or has read the material or what level of expertise this court can rely on those articles. Some are in the preparation of a learning module as attachment, not as mandatory reading. Quite clearly I can base these modules ‘go have a read of this’. I didn’t see any relevance unless you can make specific relevance to the expert evidence I have before me of those articles. You can’t and I won’t allow it.[222]

The line of questioning stalled. Even unfamiliarity with institutional training materials concerned with bias and error is not regarded as significant in relation to the credibility of the witness or the reliability of his testimony. The rules of evidence and this witness’s expressed ignorance deprived mainstream scientific materials of legal significance. We can observe the practical effects of these accusatorial practices in the judgments. The wide range of metho-dological problems raised in cross-examination are not treated as evidence and so not considered epistemologically disruptive. In a formalistic way,

rules of evidence and procedure permitted — perhaps even compelled — these judicial officers to represent the latent fingerprint examiner’s evidence

as ‘unshaken’.[223]

In the end, the best available scientific resources on latent fingerprint evidence were characterised as ‘fabulous information’,[224] ‘someone’s opin-ion’,[225] ‘documents which purport to be written by academics in the United

States’,[226] ‘arguments’,[227] ‘hypotheticals’ and ‘potentials’ of unknown prove-nance and uncertain value:[228]

Magistrate: How can I be convinced these are key scientific and international legal committees? These are reports published in a journal. I have a group of persons, scientists or otherwise that put something to the Department of Justice and I have a follow up report, a report on matters that this witness has conceded errors appear to have been made. Nothing before me that shows those errors could be attributed to exact errors in this type of matter. These are all hypotheticals, potentials, I don’t know which one of these experts can be taken as experts.

I have no judicial bodies or relevant judicial Court decisions that say that these carry any weight whether for character or otherwise. There are thousands of reports, there are thousands of researches on so many topics worldwide and I admire the amount of work that’s gone into this in terms of presenting this material but I can’t see how these type of reports and this witness’ failure to read material that is not deemed by the appropriate authorities to be specific to his work or otherwise other than in the training journal there’s some reference to some matters. How this can be held against his character or how any weight can be held in this particular matter?[229]

There was no reason to characterise the reports in such pejorative terms. They are, self-evidently, scientific reports from prestigious scientific and technical organisations authored by eminent committees of scientists (and others).[230]

In Aytugrul v The Queen, the High Court reiterated the longstanding expectation that issues with expert opinion evidence must be addressed at trial.[231] Yet, there are relatively few practical means of introducing auth-oritative reports, relevant scientific research or, scientific insights on methods and practices, in criminal proceedings.[232] Procedural difficulties are comp-ounded by shrinking legal aid budgets and the difficulty of locating approp-riately qualified, and potentially effective, rebuttal experts.[233] Decision-maker access to authoritative scientific materials should not be dependent on the training and proclivities of particular forensic practitioners, the prosecutor’s interpretation of their role, the initiative of defence counsel, or a third party’s impression of the need to commit additional public funding to provide an expert in a particular case.[234]

Our admissibility regime is currently indifferent to the reliability of scientific and technical forms of opinion evidence.[235] Commenting on civil litigation, in Dasreef Pty Ltd v Hawchar (‘Dasreef’), Heydon J insisted: ‘[T]he reasoning must be stated. The opposing party is not to be left to find out about the expert’s thinking for the first time in cross-examination.’[236] Yet that is precisely what happened in criminal proceedings in JP. Notwithstanding Dasreef, Honeysett v The Queen[237] and R v Tang,[238] our criminal courts do not actually require opinions to be based on identifiable knowledge, as opposed to belief, speculation and ipse dixit.[239]

In an age of validation studies, error rates and likelihood ratios, legal inst-itutions should no longer take the subjective opinions of latent fingerprint examiners on trust. By allowing ignorance to inoculate an expert witness against challenge, our courts deprive themselves of mainstream scientific knowledge.[240] It is neither appropriate nor rational to prefer the impressions of individual examiners to systematic scientific research on their procedures and abilities. Our concern is that important epistemological issues were not, and are not, routinely considered at trial or on appeal (or in most plea negotiations). Ironically, traditional legal practices prevented the Court from accessing knowledge on the very issues they were required to decide.[241] Namely, what was the probative value of the latent fingerprint evidence and could it support guilt beyond reasonable doubt?

VIII Conclusion

In this article, we have sought to illustrate some of the limits to cross-examination of expert evidence, and the institutional implications of those limits, through a detailed analysis of a recent trial (and appeal). Using extracts from the proceedings, we explained why cross-examination, and conventions around cross-examination, might not be a particularly effective means for elucidating knowledge or truth.[242]

In JP, rules and procedures impeded an unusually well-informed and prepared cross-examiner from penetrating the unsubstantiated beliefs and misinformation offered by an expert witness in response to a range of

well-directed questions. Many of the answers provided during cross-examination — ‘the evidence’ — were incompatible with scientific knowledge. The judicial officers who supervised and ultimately decided this case did not engage with the scientific research and reports raised by the defence. These judicial officers perceived nothing epistemologically (or institutionally) destabilising in the challenge made by the defence. The defence counsel’s attempts to introduce concerns with the reliability of the state’s evidence — including forensic science procedures and the expression of results — did not influence judicial reliance on that evidence at trial or on appeal. This article explains why existing rules and procedures do not guarantee the reliability of forensic science evidence or conditions conducive to its rational evaluation. It also illustrates how rules of evidence and procedure can prevent institutional learning. In the end, the opportunity for our criminal legal system, part-icularly the courts, to learn about the issues aired in authoritative scientific reports, and consider their implications for this and other criminal investigations and proceedings, was lost. This loss is particularly unfortunate because scientifically-informed challenges are exceptional within our criminal legal system.


* Professor, School of Law, UNSW Sydney; Research Professor (fractional), School of Law, Northumbria University. Research was supported by the Australian Research Council (LP[1]60100008). Manuscript accepted 29 August 2018.

** Associate Professor, Allard School of Law, The University of British Columbia.

† Associate Professor, School of Psychology, UNSW Sydney.

‡ Senior Lecturer, School of Law, UNSW Sydney.

[1] John Henry Wigmore, Evidence in Trials at Common Law, ed James H Chadbourn (Little, Brown and Company, rev ed, 1974) vol 5, 32.

[2] Ibid.

[3] R v JP (Children’s Court of New South Wales, Magistrate Mijovich, 27 January 2015)

(‘JP (Trial)’).

[4] JP v DPP (NSW) [2015] NSWSC 1669; (2015) 256 A Crim R 447 (‘JP (Appeal)’).

[5] See, eg, Gary Edmond and Mehera San Roque, ‘The Cool Crucible: Forensic Science and the Frailty of the Criminal Trial’ (2012) 24(1) Current Issues in Criminal Justice 51; Gary Edmond, ‘Forensic Science Evidence, Adversarial Criminal Proceedings, and Mainstream Scientific “Advice”’ in Darryl K Brown, Jenia I Turner and Bettina Weisser (eds), The Oxford Handbook of Criminal Process (Oxford University Press, 2019) 761.

[6] The vast majority of forensic science expert evidence is adduced by the state: see Ian Freckelton et al, Expert Evidence and Criminal Jury Trials (Oxford University Press, 2016)

8 [1.33].

[7] The word ‘expert’ is in scare quotes or italics to indicate that the precise status or scope of expertise can sometimes seem to be problematic.

[8] See generally Gary Edmond, David Hamer and Emma Cunliffe, ‘A Little Ignorance is a Dangerous Thing: Engaging with Exogenous Knowledge Not Adduced by the Parties’ (2016) 25(3) Griffith Law Review 383. DNA profiling and some chemistry procedures, derived from the sciences, are exceptions, having largely emerged out of mainstream scientific inquiry: National Research Council, Strengthening Forensic Science in the United States: A Path Forward (National Academies Press, 2009) 40–1 (‘NRC Report’); David A Harris, Failed Evidence: Why Law Enforcement Resists Science (New York University Press, 2012) 5. This article is primarily focused on comparison procedures.

[9] Consider the lack of engagement with scientific research across the growing number of voice comparison cases: Kheir v The Queen [2014] VSCA 200; (2014) 43 VR 308; Tran v The Queen [2016] VSCA 79; Nguyen v The Queen [2017] NSWCCA 4; (2017) 264 A Crim R 405; Chen v The Queen [2018] NSWCCA 106 (‘Chen’). Another recent example is the skilled but apparently ineffective cross-examination in Taub v The Queen [2017] NSWCCA 198; (2017) 95 NSWLR 388, 400–2 [68]–[76] (Simpson JA), discussed in Gary Edmond and D Brynn Hibbert, ‘Calibrating the Scales of Justice: Inferences on Prohibited Drug Manufacture’ (2018) 42(4) Criminal Law Journal 208, 216–31.

[10] Most of the recent challenges to forensic science evidence in Australian courts have disproportionally focused on DNA profiling and image interpretation: see, eg, Tran (1990) 50 A Crim R 233, 242–3 (McInerney J); R v Lucas [1992] VicRp 56; [1992] 2 VR 109; Pantoja (1996) 88 A Crim R 554; R v Karger [2001] SASC 64; (2001) 83 SASR 1; R v Gallagher [2001] NSWSC 462; Fitzgerald v The Queen [2014] HCA 28; (2014) 311 ALR 158; Tuite v The Queen [2015] VSCA 148; (2015) 49 VR 196 (‘Tuite’); R v Tang [2006] NSWCCA 167; (2006) 65 NSWLR 681 (‘Tang’); Murdoch v The Queen [2007] NTCCA 1; (2007) 167 A Crim R 329; Morgan v R [2011] NSWCCA 257; (2011) 215 A Crim R 33; R v Dastagir (2013) 118 SASR 83; Honeysett v The Queen (2013) 233 A Crim R 152, revd (2014) 253 CLR 122 (‘Honeysett (High Court)’).

[11] NRC Report (n 8) 7; see also at 7–8. For an introduction and overview, see Gary Edmond, ‘What Lawyers Should Know about the Forensic “Sciences”’ [2015] AdelLawRw 3; (2015) 36(1) Adelaide Law Review 33.

[12] See Harris (n 8) 24; President’s Council of Advisors on Science and Technology, Executive Office of the President, Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods (Report to the President, September 2016) 87 (‘PCAST Report’).

[13] For a comprehensive review, see Gary Edmond, ‘Latent Science: A History of Challenges to Fingerprint Evidence in Australia’ (2019) 38(2) University of Queensland Law Journal (forthcoming) (‘Latent Science’).

[14] Domain-irrelevant information is information that is not required to compare fingerprints: see below Part V(E)(1).

[15] Contrast responses from judicial officers conversant with the issues: Judge Harry T Edwards, ‘Solving the Problems That Plague the Forensic Science Community’ (2009) 50 (Fall) Jurimetrics 5; Judge Nancy Gertner, ‘Commentary on “The Need for a Research Culture in the Forensic Sciences”’ (2011) 58(3) UCLA Law Review 789; Justice Ian Binnie, ‘Science in the Courtroom: The Mouse That Roared’ (2007) 56 University of New Brunswick Law

Journal 307; Chris Maxwell, ‘Preventing Miscarriages of Justice: The Reliability of Forensic Evidence and the Role of the Trial Judge as Gatekeeper’ (2019) 93(8) Australian Law Journal 642.

[16] See generally Kristy A Martire and Gary Edmond, ‘Rethinking Expert Opinion Evidence’ [2017] MelbULawRw 14; (2017) 40(3) Melbourne University Law Review 967.

[17] Although, the record could hardly be considered public. This is yet another problem with transparency, public accountability and feedback for our legal institutions.

[18] See Geraldine MacKenzie, Andrew Vincent and John Zeleznikow, ‘Negotiating about Charges and Pleas: Balancing Interests and Justice’ (2015) 24(4) Group Decision and Negotiation 577; Australian Institute of Criminology, Plea Negotiations: An Empirical Analysis (Trends and Issues in Crime and Criminal Justice No 544, April 2018) 1.

[19] Here, there is the risk of victim blaming, to the extent that a defendant is deemed responsible for tactical decisions.

[20] To use the words of Albert W Alschuler (albeit in the context of plea bargains), such mistakes or failures of experience represent ‘tactical decision[s] irrelevant to any proper objective of criminal proceedings’: Albert W Alschuler, ‘Implementing the Criminal Defendant’s Right to Trial: Alternatives to the Plea Bargaining System’ (1983) 50(3) University of Chicago Law Review 931, 932.

[21] Some proportion of these clients will have been innocent.

[22] See Matthew B Thompson, Jason M Tangen and Duncan J McCarthy, ‘Expertise in Fingerprint Identification’ (2013) 58(6) Journal of Forensic Sciences 1519; Jason M Tangen, Matthew B Thompson and Duncan J McCarthy, ‘Identifying Fingerprint Expertise’ 22(8) Psychological Science 995.

[23] The fingerprint examiner was using a standardised reporting template: see Gary Edmond, Kristy Martire and Mehera San Roque, ‘Expert Reports and the Forensic Sciences’ [2017] UNSWLawJl 22; (2017) 40(2) University of New South Wales Law Journal 590, 602–3, 627–37.

[24] Davie v The Lord Provost, Magistrates and Councillors of the City of Edinburgh 1953 SC 34, 40 (Lord President Cooper) (‘Davie’); Makita (Australia) Pty Ltd v Sprowles [2001] NSWCA 305; (2001) 52 NSWLR 705, 741 [81] (Heydon JA) (‘Makita’), quoting Pollock v Wellington (1996) 15 WAR 1, 3–4 (Anderson J); Gary Edmond, ‘Forensic Science Evidence and the Conditions for Rational (Jury) Evaluation’ [2015] MelbULawRw 17; (2015) 39(1) Melbourne University Law Review 77.

[25] In Honeysett (High Court) (n 10), the High Court accepted the definition of ‘knowledge’ as ‘acquaintance with facts, truths, or principles, as from study or investigation’: at 131 [23] (French CJ, Kiefel, Bell, Gageler and Keane JJ) (emphasis omitted).

[26] Or, expressing the terminology of IMM v The Queen [2016] HCA 14; (2016) 257 CLR 300 (‘IMM’) — it was weaker than suggested by the witness.

[27] Patrick Devlin, The Judge (University of Chicago Press, 1981) 60–1:

The English say that the best way of getting at the truth is to have each party dig for the facts that help it; between them they will bring all to light. ... [T]wo prejudiced searchers starting from opposite ends of the field will between them be less likely to miss anything than the impartial searcher starting in the middle.

Cf John H Langbein, The Origins of Adversary Criminal Trial (Oxford University Press,

2003) 8:

The development of adversary criminal trial raised an acute theoretical challenge, which has never been satisfactorily resolved in the Anglo-American tradition: how to justify the truth-impairing tendencies of a procedure that remits to partisans the work of gathering and presenting the evidence upon which accurate adjudication depends.

[28] Boucher v The Queen [1955] SCR 16, 23–4 (Rand J).

[29] In recent years, new expectations have been placed on defendants, notably in relation to alibis, calling expert evidence and silence: see, eg, Evidence Amendment (Evidence of Silence) Act 2013 (NSW); Criminal Procedure Amendment (Mandatory Pre-Trial Defence Disclosure) Act 2013 (NSW).

[30] See Gary Edmond et al, ‘Contextual Bias and Cross-Contamination in the Forensic Sciences: The Corrosive Implications for Investigations, Plea Bargains, Trials and Appeals’ (2015) 14(1) Law, Probability & Risk 1, 20–1; John B Bishop, Prosecution without Trial (Butterworths, 1989) 184.

[31] Daubert v Merrell Dow Pharmaceuticals Inc, [1993] USSC 99; 509 US 579, 596 (Blackmun J for Blackmun, White, O’Connor, Scalia, Kennedy, Souter and Thomas JJ) (1993) (‘Daubert’). The formal requirement of ‘reliability’ and willingness to admit ‘shaky’ evidence sit uncomfortably together in Daubert. Though, we should not forget that Daubert was an appeal from a civil case. See also Melendez-Diaz v Massachusetts, 557 US 305, 305–6, 309–11 (Scalia J for Scalia, Stevens, Souter, Thomas and Ginsburg JJ) (2009).

[32] See Wigmore (n 1) 32; Edmond and San Roque (n 5) 51–3.

[33] See NRC Report (n 8) 53.

[34] Dawn McQuiston-Surrett and Michael J Saks, ‘The Testimony of Forensic Identification Science: What Expert Witnesses Say and What Factfinders Hear’ (2009) 33(5) Law and Human Behavior 436, 439 (citations omitted). See also Margaret Bull Kovera et al, ‘Expert Testimony in Child Sexual Abuse Cases: Effects of Expert Testimony and Cross-Examination’ (1994) 18(6) Law and Human Behavior 653.

[35] Law Commission, Expert Evidence in Criminal Proceedings in England and Wales (Report No 325, 21 March 2011) 5 [1.20] (citations omitted); see also at 6 [1.24]. See also NRC Report

(n 8) 53.

[36] Michael Lynch and David Bogen, The Spectacle of History: Speech, Text, and Memory at the Iran–Contra Hearings (Duke University Press, 1996) 151–3; Jules Epstein, ‘The Great Engine That Couldn’t: Science, Mistaken Identifications, and the Limits of Cross-Examination’ (2007) 36(3) Stetson Law Review 727.

[37] R v Parker [1912] ArgusLawRp 24; [1912] VLR 152, 158 (Hodges J) (‘Parker’). See also R v Blacker [1910] NSWStRp 27; (1910) 10 SR (NSW) 357, 360 (Cullen CJ).

[38] For those interested in developments in other jurisdictions, see Gary Edmond et al, ‘Admissibility Compared: The Reception of Incriminating Expert Evidence (ie, Forensic Science) in Four Adversarial Jurisdictions’ (2013) 3 University of Denver Criminal Law Review 31.

[39] See, eg, Parker (n 37). The question on reserve, confirmed by the majority, was whether a single latent fingerprint match could support guilt. The Chief Justice dissented: at 153–6 (Madden CJ); however, all of the judges agreed that the claim that fingerprints are unique should not have been admitted: at 155 (Madden CJ), 158 (Hodges J), 159 (Cussen J).

See also Jeremy Gans, ‘A Tale of Two High Court Forensic Cases’ [2011] SydLawRw 22; (2011) 33(3) Sydney Law Review 515.

[40] See, eg, People v Jennings, 96 NE 1077, 1081–2 (Ill Sup Ct, 1911). For historical accounts of fingerprinting, see Simon A Cole, Suspect Identities: A History of Fingerprinting and Criminal Identification (Harvard University Press, 2001); Chandak Sengoopta, Imprint of the Raj: How Fingerprinting Was Born in Colonial India (Macmillan, 2003).

[41] Cf Michael J Saks and Jonathan J Koehler, ‘The Individualization Fallacy in Forensic Science Evidence’ (2008) 61(1) Vanderbilt Law Review 199; Simon A Cole, ‘Individualization is Dead, Long Live Individualization! Reforms of Reporting Practices for Fingerprint Analysis in the United States’ (2014) 13(2) Law, Probability & Risk 117.

[42] See Edmond, ‘Latent Science’ (n 13).

[43] NRC Report (n 8) 53, 142–3, 184. See also PCAST Report (n 12) 87, 91–5. See, eg, Bradford T Ulery et al, ‘Repeatability and Reproducibility of Decisions by Latent Fingerprint Examiners’ (2012) 7(3) PLoS One e32800:1–12; Tangen, Thompson and McCarthy (n 22).

[44] NRC Report (n 8) 137. See also RA Huber, ‘Expert Witnesses: In Defence of Expert Witnesses in General and of Document Examiners in Particular’ (1959) 2(3) Criminal Law Quarterly 276. See generally David R Ashbaugh, Quantitative–Qualitative Friction Ridge Analysis: An Introduction to Basic and Advanced Ridgeology (CRC Press, 1999).

[45] See Simon A Cole and Andrew Roberts, ‘Certainty, Individualisation and the Subjective Nature of Expert Fingerprint Evidence’ [2012] (11) Criminal Law Review 824.

[46] We do not suggest that there is a problem with subjective analyses per se, but practices need to be standardised and risks managed. See generally Expert Working Group on Human Factors in Latent Print Analysis, National Institute of Standards and Technology, Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach (Report, February 2012) (‘NIST Report’).

[47] See, eg, Transcript of Proceedings, Forbes v The Queen [2010] HCATrans 120, discussed in Gans (n 39) 531–6. See also David Kaye, The Double Helix and the Law of Evidence (Harvard University Press, 2010).

[48] See Harris (n 8) 24–30.

[49] See, eg, Bennett v Police (SA) (2005) 239 LSJS 465 (‘Bennett’); Hillstead v The Queen [2005] WASCA 116; R v SMR [2002] NSWCCA 258; Mickelberg v The Queen (2004) 29 WAR 13; R v Lawless [1974] VicRp 49; [1974] VR 398; R v O’Callaghan [1976] VicRp 72; [1976] VR 676. Cf R v Smith [2011] 2 Cr App R 16.

[50] Ghebrat v The Queen (2011) 214 A Crim R 140, 143 [15], 145 [25] (Tate JA) (‘Ghebrat’).

[51] The only time questions were raised was in the dissenting opinion of Madden CJ in Parker

(n 37) 154–5. See also obiter in Tang (n 10) 713–15 [145]–[155] (Spigelman CJ).

[52] See, eg, Parker (n 37). See also the diachronic response to bite mark evidence discussed in D Michael Risinger, ‘Navigating Expert Reliability: Are Criminal Standards of Certainty Being Left on the Dock?’ (2000) 64(1) Albany Law Review 99, 135–43.

[53] Bennett (n 49) 465 [5] (Doyle CJ). And, that the examiner ‘was qualified to express the opinion that he gave’: at 465 [5] (Doyle CJ); that the two fingerprints were ‘identical, and came from the same person’: at 465 [6] (Doyle CJ).

[54] This is unfortunate because courts are not in a position to test or lend meaningful epistemic imprimatur: see above n 9; IMM (n 26) 315 [52] (French CJ, Kiefel, Bell and Keane JJ).

[55] So-called ‘co-production’: see Sheila Jasanoff (ed), States of Knowledge: The Co-Production of Science and the Social Order (Routledge, 2004).

[56] Even using reliability-based admissibility standards: see Gary Edmond, ‘Specialised Knowledge, the Exclusionary Discretions and Reliability: Reassessing Incriminating

Expert Opinion Evidence’ [2008] UNSWLawJl 1; (2008) 31(1) University of New South Wales Law Journal 1 (‘Specialised Knowledge’).

[57] Tang (n 10) 712 [137] (Spigelman CJ); Chen (n 9) [62] (Hoeben CJ at CL, Schmidt and Campbell JJ). The only real exception was the Victorian Court of Appeal briefly requiring trial judges to consider reliability under s 137 of the Evidence Act 2008 (Vic) (before the High Court decision in IMM (n 26)) in Tuite (n 10) 200–1 [10] (Maxwell ACJ, Redlich and Weinberg JJA). Cf IMM (n26), discussed in Gary Edmond, ‘Icarus and the Evidence Act: Section 137, Probative Value and Taking Forensic Science Evidence “at Its Highest”’ [2017] MelbULawRw 22; (2017) 41(1) Melbourne University Law Review 106 (‘Icarus and the Evidence Act’).

[58] See, eg, Bennett (n 49) 474–5 [54]–[56] (Doyle CJ). But see Makita (n 24); Dasreef Pty Ltd v Hawchar [2011] HCA 21; (2011) 243 CLR 588, 622–3 [91], 628 [101], 636 [123] (Heydon J) (‘Dasreef’); Davie (n 24) 40 (Lord President Cooper) (in a different jurisdiction). Note the reference to the plurality’s judgment in Dasreef in JP (Appeal) (n 4) 455–6 [32]–[33] (Beech-Jones J), quoting Dasreef (n 58) 604 [37] (French CJ, Gummow, Hayne, Crennan, Kiefel and Bell JJ). See generally Gary Edmond and Kristy A Martire, ‘Knowing Experts? Section 79, Forensic Science Evidence and the Limits of “Training, Study or Experience”’ in Andrew Roberts and Jeremy Gans (eds), Critical Perspectives on the Uniform Evidence Law (Federation Press, 2017) 80.

[59] See Martire and Edmond, ‘Rethinking Expert Opinion Evidence’ (n 16). See also Edmond and Martire (n 58).

[60] JP (Appeal) (n 4) 449 [1] (Beech-Jones J); Crimes Act 1900 (NSW) ss 112(1)–(2).

[61] JP (Appeal) (n 4) 450 [8] (Beech-Jones J).

[62] Ibid 450 [9] (Beech-Jones J).

[63] ‘A preliminary examination of the prints led to the police forming a suspicion that JP and another male committed the break-in’: ibid 450 [9] (Beech-Jones J).

[64] The suspected co-offender was arrested and charged on 16 October 2016.

[65] See Evidence Act 1995 (NSW) s 177.

[66] JP (Appeal) (n 4) 450–1 [11] (Beech-Jones J).

[67] The redacted Expert Certificate is reproduced as an appendix to Edmond, Martire and San Roque (n 23) 627.

[68] Ibid 629. For reasons that were never made entirely clear, he was not the first or only latent fingerprint examiner to analyse and compare the latent print with JP’s prints: see Transcript of Proceedings, R v JP (Children’s Court of New South Wales, Magistrate Mijovich,

27 January 2015) 10 (‘JP (Trial Transcript) (27 January)’). Precisely why so many different people were involved and who evaluated and who ‘verified’ the result are unclear. Backstage practices are not transparent and it is impossible to understand exactly how ACE-V is practised from the Expert Certificate and testimony.

[69] JP (Appeal) (n 4) 450–1 [11] (Beech-Jones J).

[70] JP (Trial Transcript) (27 January) (n 68) 30.

[71] Ibid.

[72] Part V of the Crimes (Appeal and Review) Act 2001 (NSW) provides for an appeal as of right on matters of law, and requires leave for appeal on matters of fact, or mixed law and fact:

at ss 523.

[73] JP (Appeal) (n 4) 460–8 [50]–[91] (Beech-Jones J).

[74] Ibid 468 [93].

[75] Ibid 463 [62]–[63]. It is very unlikely that these would have made any difference, for the reasons explained in Edmond, ‘Icarus and the Evidence Act’ (n 57).

[76] NRC Report (n 8).

[77] NIST Report (n 46).

[78] Sir Anthony Campbell, The Fingerprint Inquiry Report (Report, December 2011) (‘SFI Report’).

[79] NRC Report (n 8) xix.

[80] Ibid iii.

[81] ‘About NIST’, NIST (Web Page, 14 June 2017) <https://www.nist.gov/about-nist>, archived at <https://perma.cc/L6LQ-2CHR>.

[82] SFI Report (n 78) 5.

[83] NRC Report (n 8) 137–9 (emphasis omitted) (citations omitted). See also PCAST Report

(n 12) 88–90; William Thompson et al, American Association for the Advancement of Science, Forensic Science Assessments: A Quality and Gap Analysis — Latent Fingerprint Examination (Report No 2, 15 September 2017) 95–8 (‘AAAS Report’).

[84] See also Glenn Langenberg, ‘A Performance Study of the ACE-V Process: A Pilot Study to Measure the Accuracy, Precision, Reproducibility, Repeatability, and Biasability of Conclusions Resulting from the ACE-V Process’ (2009) 59(2) Journal of Forensic Identification 219.

[85] NRC Report (n 8) 142–3 (citations omitted), quoted in NIST Report (n 46) 9, 124.

[86] Bradford T Ulery et al, ‘Accuracy and Reliability of Forensic Latent Fingerprint Decisions’ (2011) 108(19) Proceedings of the National Academy of Sciences of the United States of America 7733; Ulery et al, ‘Repeatability and Reproducibility of Decisions by Latent Fingerprint Examiners’ (n 43); Tangen, Thompson and McCarthy (n 22).

[87] See below Part V(C)(1).

[88] NIST Report (n 46) 8.

[89] Ibid 204.

[90] JP (Trial Transcript) (27 January) (n 68) 12.

[91] See Bradford T Ulery et al, ‘Changes in Latent Fingerprint Examiners’ Markup between Analysis and Comparison’ (2015) 247 Forensic Science International 54.

[92] Transcript of Proceedings, R v JP (Children’s Court of New South Wales, Magistrate Mijovich, 13 January 2015) 33 (‘JP (Trial Transcript) (13 January)’).

[93] NRC Report (n 8) 143 (emphasis added); see also at 142–5, 149.

[94] See generally Linda T Kohn, Janet M Corrigan and Molla S Donaldson (eds), To Err Is Human: Building a Safer Health System (National Academy Press, 2000). See also Itiel E Dror and Simon A Cole, ‘The Vision in “Blind” Justice: Expert Perception, Judgment, and Visual Cognition in Forensic Pattern Recognition’ (2010) 17(2) Psychonomic Bulletin & Review 161.

[95] The NIST Report (n 46), as its full title implies (Latent Print Examination and Human Factors: Improving the Practice through a Systems Approach), was entirely oriented to addressing risks raised by human factors.

[96] NRC Report (n 8) 184; see also at 122.

[97] See ibid 183–4; PCAST Report (n 12) 95–6.

[98] See David L Faigman, John Monahan and Christopher Slobogin, ‘Group to Individual (G2i) Inference in Scientific Expert Testimony’ (2014) 81(2) University of Chicago Law Review 417, 453–6.

[99] PCAST Report (n 12).

[100] Ibid 101–2; NRC Report (n 8) 142–3.

[101] PCAST Report (n 12) 87–97.

[102] See, eg, Ulery et al, ‘Accuracy and Reliability of Forensic Latent Fingerprint Decisions’

(n 86).

[103] PCAST Report (n 12) 9–10 (emphasis added) (citations omitted); see also at 87–103. The PCAST Report stressed: ‘Examiners have sometimes testified, for example, that their conclusions are “100 percent certain;” or have “zero,” “essentially zero,” or “negligible,” error rate. ... [H]owever, such statements are not scientifically defensible: all laboratory tests and feature-comparison analyses have non-zero error rates’: at 3. See also NRC Report (n 8)

142–5; NIST Report (n 46) 21–38; SFI Report (n 78) 602–32.

[104] PCAST Report (n 12) 96.

[105] NRC Report (n 8) 143.

[106] Ibid 142.

[107] JP (Trial Transcript) (27 January) (n 68) 12.

[108] Ibid 11.

[109] Ibid 12–13.

[110] Ibid 16.

[111] See Edmond, Hamer and Cunliffe (n 8).

[112] JP (Trial Transcript) (27 January) (n 68) 12.

[113] Ibid 19–20.

[114] Brandon Mayfield was implicated in the 2004 Madrid train bombings based on a mistaken fingerprint ‘match’ provided by the FBI: Office of the Inspector General, US Department of Justice, A Review of the FBI’s Handling of the Brandon Mayfield Case (Report, March 2006). Shirley McKie was a Scottish detective accused of perjury on the basis of mistaken fingerprint evidence. The events led to the Scottish Fingerprint Inquiry: SFI Report (n 78).

[115] Transcript of Proceedings, R v JP (Children’s Court of New South Wales, Magistrate Mijovich, 15 January 2015) 24 (‘JP (Trial Transcript) (15 January)’).

[116] JP (Trial Transcript) (13 January) (n 92) 38; see also at 36.

[117] JP (Trial Transcript) (27 January) (n 68) 11–12. The reason is because we do not know the ground truth.

[118] See Rachel A Searston and Jason M Tangen, ‘The Emergence of Perceptual Expertise with Fingerprints over Time’ (2017) 6(4) Journal of Applied Research in Memory and Cognition 442. In some domains, performance accuracy and years of experience are not strongly correlated: see, eg, David White et al, ‘Passport Officers’ Errors in Face Matching’ (2014) 9(8) Plos One e103510:1–6.

[119] We do not contend that there was any disagreement in JP, but then there would probably be no official record (or disclosure, at least) if there had been.

[120] The AAAS Report (n 83) endorses the need for further research on possible errors by examiners in the ‘short run’, until ‘quantitative methods for estimating the probative value or weight of fingerprint evidence’ are developed: at 5, 22.

[121] NRC Report (n 8) 142, quoting Jennifer L Mnookin, ‘The Validity of Latent Fingerprint Identification: Confessions of a Fingerprinting Moderate’ (2008) 7(2) Law, Probability & Risk 127, 139.

[122] NIST Report (n 46) 72.

[123] SFI Report (n 78) 740.

[124] AAAS Report (n 83) 9.

[125] JP (Trial Transcript) (13 January) (n 92) 13–14; see also at 25–6.

[126] These forms of expression are much more problematic than the notorious prosecutor’s fallacy, whereby the probability of the evidence given the hypothesis is mistaken for the probability of the hypothesis given the evidence: see William C Thompson and Edward L Schumann, ‘Interpretation of Statistical Evidence in Criminal Trials: The Prosecutor’s Fallacy and the Defense Attorney’s Fallacy’ (1987) 11(3) Law and Human Behavior 167.

[127] JP (Trial Transcript) (27 January) (n 68) 17, quoting NIST Report (n 46) 72 (Recommendation 3.7). See also SFI Report (n 78) 740. This testimony (and the expression) is also inconsistent with the Australian standards for reporting forensic science testimony: Standards Australia, Forensic Analysis — Part 4: Reporting (Standard No AS 5883.4—2013, 2 May 2013).

[128] JP (Trial Transcript) (27 January) (n 68) 17.

[129] His position is also inconsistent with the (recently revised) advice of the International Association for Identification — a society established by fingerprint examiners and others more than a century earlier: Standardization II Committee, The Report of the International Association for Identification, Standardization II Committee (Report, 30 September 2010) 19.

[130] See, eg, New South Wales Law Reform Commission, Expert Witnesses (Report No 109, June 2005) 70–5; White Burgess Langille Inman v Abbott & Haliburton Co Ltd [2015] 2 SCR 182.

[131] NRC Report (n 8) 9; see also at 43, 87, 111.

[132] PCAST Report (n 12) 31. See also NIST Report (n 46) 43–4.

[133] Blinding protects participants, investigators, analysts and reviewers from information and suggestions that might influence their judgments (eg another analyst’s conclusion). See Itiel E Dror et al, ‘Letter to the Editor — Context Management Toolbox: A Linear Sequential Unmasking (LSU) Approach to Minimizing Cognitive Bias in Forensic Decision Making’ (2015) 60(4) Journal of Forensic Sciences 1111.

[134] NIST Report (n 46) 40.

[135] Ibid 44 (Recommendation 3.3). See also the discussion on the impact that extraneous information can have on individuals: at 10–12. See also SFI Report (n 78) 741 (Recommendations 6–8).

[136] PCAST Report (n 12) 10.

[137] AAAS Report (n 83) 7–8, 35–6.

[138] National Commission on Forensic Science, Reflecting Back: Looking toward the Future (Report, 11 April 2017) 1–2, 6. See also NRC Report (n 8) 191.

[139] National Commission on Forensic Science, Ensuring That Forensic Analysis Is Based upon Task-Relevant Information (Adopted Views of the Commission, 8 December 2015) 1–2. See also PCAST Report (n 12) 10–12.

[140] NIST Report (n 46) 97.

[141] Ibid.

[142] JP (Trial Transcript) (27 January) (n 68) 4.

[143] Ibid 10–11.

[144] Ibid.

[145] The NRC Report (n 8) warned that ‘ACE-V does not guard against bias’: at 142.

[146] A point partially addressed by linear ACE-V, which was not used in this case: see PCAST Report (n 12) 101.

[147] Itiel E Dror and David Charlton, ‘Why Experts Make Errors’ (2006) 56(4) Journal of Forensic Identification 600; Itiel E Dror, David Charlton and Ailsa E Péron, ‘Contextual Information Renders Experts Vulnerable to Making Erroneous Identifications’ (2006) 156(1) Forensic Science International 74.

[148] JP (Trial Transcript) (27 January) (n 68) 5–7.

[149] See above n 147.

[150] JP (Trial Transcript) (27 January) (n 68) 5.

[151] Ibid 4, 9–11.

[152] See Saks and Koehler (n 41) 206 nn 29–35.

[153] PCAST Report (n 12) 62 (emphasis in original). See also NRC Report (n 8) 143–5.

[154] AAAS Report (n 83) 5; see also at 21.

[155] See Saks and Koehler (n 41).

[156] JP (Trial Transcript) (13 January) (n 92) 24; see also at 25.

[157] Ibid 26.

[158] Ibid 35–6.

[159] See generally Jonathan J Koehler and Michael J Saks, ‘Individualization Claims in Forensic Science: Still Unwarranted’ (2010) 75(4) Brooklyn Law Review 1187.

[160] The failure to disclose such differences in reports was also excused in Bennett (n 49): at

474–5 [54]–[58] (Doyle CJ). There is a misguided tendency to describe the latent

and reference prints as identical. There are always differences.

[161] JP (Trial Transcript) (15 January) (n 115) 4–5; see also at 6–9, 12–14. See also JP (Trial Transcript) (13 January) (n 92) 33–4.

[162] PCAST Report (n 12) 5–6, 143.

[163] See JP (Trial Transcript) (13 January) (n 92) 24–6. Historians and philosophers of science have explained how metaphysical commitments structure the ways in which one conceptualises. For influential work in this area, see Thomas S Kuhn, The Copernican Revolution: Planetary Astronomy in the Development of Western Thought (Harvard University Press, 1957); E A Burtt, The Metaphysical Foundations of Modern Science (Dover Publications, 2003).

[164] JP (Trial Transcript) (27 January) (n 68) 5.

[165] Ibid 13.

[166] Ibid 18.

[167] Ibid 14.

[168] Ibid 17.

[169] Edmond, Martire and San Roque (n 23) 627.

[170] See generally Thomas F Gieryn, Cultural Boundaries of Science: Credibility on the Line (University of Chicago Press, 1999).

[171] JP (Trial Transcript) (27 January) (n 68) 17.

[172] Ibid 15. Consider the expert testimony admitted in Commissioner for Government

Transport v Adamcik [1961] HCA 43; (1961) 106 CLR 292, despite the cross-examination revealing the expert’s limited knowledge and unsupported claims: at 303 (Menzies J).

[173] Uniform Civil Procedure Rules 2005 (NSW) sch 7 (‘Expert Witness Code of Conduct’).

[174] Edmond, Martire and San Roque (n 23) 628.

[175] JP (Trial Transcript) (27 January) (n 68) 13–14; see also at 25. See also JP (Trial Transcript) (13 January) (n 92) 25.

[176] Expert Witness Code of Conduct (n 174) s 3(j). On the implications for admissibility, see Wood v The Queen [2012] NSWCCA 21; (2012) 84 NSWLR 581, 619–20 [728]–[731] (McClellan CJ at CL); Chen (n 9) [20]–[23], [34], [72] (Hoeben CJ at CL, Schmidt and Campbell JJ).

[177] The witness seems to believe that it is the responsibility of the defence to identify limitations and uncertainties.

[178] See Supreme Court of Victoria, Practice Note No 2 of 2014: Expert Evidence in Criminal Trials, June 2014, 4 [4.2]. It requires the expert to disclose disagreement and controversies.

[179] JP (Trial) (n 3).

[180] JP (Trial Transcript) (27 January) (n 68) 28–30. See also Bennett (n 49), where the Magistrate’s superficial engagement with the expert witness supposedly ‘exposed [the expert’s] method of working’: at 467 [16] (Doyle CJ); and ‘fully informed [the Magistrate] of the reasoning process’: at 474 [54] (Doyle CJ).

[181] JP (Trial Transcript) (27 January) (n 68) 30.

[182] Ibid 28–9 (emphasis added).

[183] Ibid 29 (emphasis added).

[184] Ibid 30.

[185] Ibid 29.

[186] Ibid 30.

[187] Ibid 29.

[188] Ibid. See also JP (Trial Transcript) (13 January) (n 92) 33.

[189] JP (Trial Transcript) (27 January) (n 68) 29.

[190] Ibid.

[191] Ibid.

[192] Ibid (emphasis added). There was a single reference to a decision by an American Court: at 30 (although the case is not cited, his Honour is presumably referring to Judge Thorne’s judgment in Utah v Quintana, 103 P 3d 168, 170–1 (Utah App, 2004)).

[193] JP (Trial Transcript) (27 January) (n 68) 29.

[194] Ibid.

[195] Ibid 30.

[196] Ibid 29.

[197] PCAST Report (n 12) 34–5, 65–6.

[198] General studies do not prevent medical doctors from treating individual patients. Rather, knowledge of the general informs particular treatment decisions.

[199] See below Part VII(A).

[200] See Kaye N Ballantyne, Gary Edmond and Bryan Found, ‘Peer Review in Forensic Science’ (2017) 277 Forensic Science International 66.

[201] See above Part V(E)(1).

[202] JP (Trial Transcript) (27 January) (n 68) 29. A similar expectation was present in Castleton (1909) 3 Cr App R 74, where Darling J asked: ‘Can the prisoner find anybody asked whose fingerprints are exactly like his?’: at 74. In Parker (n 37), Madden CJ indicated, apologetically, that the question posed by Darling J ‘cannot have been seriously meant’: at 155.

[203] JP (Trial Transcript) (27 January) (n 68) 30. This is interesting because the same position was not accepted in Parker (n 37) 155 (Madden CJ), 158 (Hodges J), 159 (Cussen J) or Ghebrat

(n 50) 145–6 [27]–[33] (Tate JA). Common law and statutory judicial notice doctrines are unduly onerous, requiring ‘knowledge that is not reasonably open to question’: Evidence Act 1995 (Cth) s 144. See generally David Hamer and Gary Edmond, ‘Judicial Notice: Beyond Adversarialism and into the Exogenous Zone’ (2016) 25(3) Griffith Law Review 291.

[204] JP (Trial Transcript) (27 January) (n 68) 30.

[205] Testimony reveals that the latent print is not identical to the reference print. This is always the case.

[206] See generally Gary Edmond et al, ‘Thinking Forensics: Cognitive Science for Forensic Practitioners’ (2017) 57(2) Science and Justice 144.

[207] JP (Trial Transcript) (27 January) (n 68) 30.

[208] JP (Appeal) (n 4) 468 [90] (Beech-Jones J).

[209] Ibid. Of interest, there was no testimony that the prints were identical — rather, there were differences. The question was whether these non-identical fingerprints were produced by the same person: at 451–2 [14]–[21] (Beech-Jones J).

[210] Gary Edmond et al, ‘How to Cross-Examine Forensic Scientists: A Guide for Lawyers’ (2014) 39(2) Australian Bar Review 174.

[211] JP (Trial Transcript) (27 January) (n 68) 5–6. The NSW Police revised their reports in the aftermath of JP, but they do not provide any indication of error or convey the magnitude of risks. There has been legitimate debate over the appropriate way to report error and the applicability of studies to actual practice. These debates raise important issues, but the main point is the necessity of a good faith attempt to present an indicative error rate based on empirical studies rather than impressions, or the results of commercial proficiency tests. See PCAST Report (n 12) 10, 17, 55, 57–9; see especially at 57 n 133. To their credit, drawing on the PCAST Report, the Victorian Police report an indicative error rate of 0.17%. See also Edmond, Martire and San Roque (n 23).

[212] JP (Trial Transcript) (27 January) (n 68) 29.

[213] See Woolmington v DPP (UK) [1935] AC 462; Azzopardi v The Queen [2001] HCA 25; (2001) 205 CLR 50;

X7 v Australian Crime Commission [2013] HCA 29; (2013) 248 CLR 92.

[214] There may be a few exceptions, such as where there is a ‘rock solid’ alibi or a DNA exoneration, but these cases will almost never be prosecuted. It is useful to consider the case of Jama in this context: FHR Vincent, Inquiry into the Circumstances that Led to the Conviction of Mr Farah Abdulkadir Jama (Report, May 2010) 32–8. See also Stephen T Goudge, Inquiry into Pediatric Forensic Pathology in Ontario (Report, 30 September 2008).

[215] See Department of Health (WA), Ross Inquiry into PathWest Laboratory Medicine WA (Final Report, 2017).

[216] See generally Antony Duff et al (eds), The Trial on Trial: Towards a Normative Theory of the Criminal Trial (Hart Publishing, 2007) vol 3, ch 3. For an argument focused specifically on forensic science evidence, see Gary Edmond and Andrew Roberts, ‘Procedural Fairness, the Criminal Trial and Forensic Science and Medicine’ [2011] SydLawRw 17; (2011) 33(3) Sydney Law Review 359.

[217] See, eg, NRC Report (n 8) 122. ‘The level of accuracy of an analysis is likely to be a key determinant of its ultimate probative value.’: at 184. See also PCAST Report (n 12) 9–10, 143.

[218] See Faigman, Monahan and Slobogin (n 98).

[219] We do not, for example, allow biologists to positively identify persons with DNA evidence. They are limited to probabilities derived from population genetics.

[220] JP (Trial Transcript) (27 January) (n 68) 15. See also R v Bornyk (2015) 320 CCC (3d) 393.

[221] See above Part V(E)(2); n 149 and accompanying text.

[222] JP (Trial Transcript) (27 January) (n 68) 8.

[223] Ibid 29.

[224] Ibid 7.

[225] Ibid 17.

[226] Ibid 21.

[227] Ibid 29.

[228] Ibid 22.

[229] Ibid. See also JP (Trial Transcript) (13 January) (n 92) 31.

[230] Interestingly, other parts of the Evidence Act 1995 (NSW) enable courts to accept (sometimes provisionally) the nature of documents on the basis of their face value: see, eg, at ss 578.

[231] [2012] HCA 15; (2012) 247 CLR 170, 183–4 [20]–[24] (French CJ, Hayne, Crennan and Bell JJ), 199–200 [66]–[67] (Heydon J). This was in response to an appellate judge referring to journal articles on appeal: Aytugrul v The Queen [2010] NSWCCA 272; (2010) 205 A Crim R 157, 174–7 [89]–[102] (McClellan CJ at CL). See also McGregor v McGregor [2012] FamCAFC 69; (2012) 47 Fam LR 498.

[232] An expert witness is generally required.

[233] One alternative strategy involves a defendant relying on a meta-expert — a witness who may not be a fingerprint examiner but who is familiar with the relevant research and research methods. Such witnesses will be in a position to discuss scientific materials, and might provide some kind of assistance in these cases, but they are unlikely to make much of a difference in practice: see Michael Lynch and Simon Cole, ‘Science and Technology Studies on Trial: Dilemmas of Expertise’ (2005) 35(2) Social Studies of Science 269; Simon A Cole, ‘A Cautionary Tale about Cautionary Tales about Intervention’ (2009) 16(1) Organization 121.

[234] See generally Keith A Findley, ‘Innocents at Risk: Adversary Imbalance, Forensic Science, and the Search for Truth’ (2008) 38(3) Seton Hall Law Review 893.

[235] See Edmond, ‘Specialised Knowledge’ (n 56); Gary Edmond, ‘The Admissibility of Forensic Science and Medicine Evidence under the Uniform Evidence Law’ (2014) 38(3) Criminal Law Journal 136; Edmond, ‘Icarus and the Evidence Act’ (n 57).

[236] Dasreef (n 58) 623 [91] (Heydon J) (citations omitted). See also Makita (n 24) 741 [81] (Heydon JA).

[237] Honeysett (High Court) (n 10).

[238] Tang (n 10).

[239] See IMM (n 26). We might note that the witness is not entitled to offer opinion, but rather only opinion based on ‘specialised knowledge’: see Tang (n 10); Honeysett (High Court)

(n 10). To the extent that our trial system disregards ‘knowledge’, safeguards may have

little value.

[240] See Edmond, Hamer and Cunliffe (n 8); Gary Edmond, ‘Expert Evidence and the Responsibilities of Prosecutors’ in Jill Hunter et al (eds), The Integrity of the Criminal Process: From Theory into Practice (Hart Publishing, 2016) 225.

[241] See, eg, Emma Cunliffe and Gary Edmond, ‘Reviewing Wrongful Convictions in Canada’ (2017) 64(3– 4) Criminal Law Quarterly 473. Recently, the Royal Society of London and the Royal Society of Edinburgh collaborated to produce legally-oriented reviews of controversial areas of forensic ‘science’: see, eg, Royal Society and Royal Society of Edinburgh, Forensic Gait Analysis: A Primer for Courts (Report, November 2017). See also Colin Aitken, Paul Roberts and Graham Jackson, Fundamentals of Probability and Statistical Evidence in Criminal Proceedings: Guidance for Judges, Lawyers, Forensic Scientists and Expert Witnesses (Practitioner Guide No 1, November 2010).

[242] We should be cautious about claims made by legally-trained individuals based on their non-systematic experience(s).