Neill, Christine; Leigh, Andrew --- "Do Gun Buy-backs Save Lives? Evidence from Time Series Variation" [2008] CICrimJust 22; (2008) 20(2) Current Issues in Criminal Justice 145
[∗] Corresponding author. Department of Economics, Wilfrid Laurier University, 75 University Ave West, Waterloo, ON, N2L 3C5, Canada. Email cneill@wlu.ca. Website: www.wlu.ca/sbe/cneill
[**] Research School of Social Sciences, Australian National University, Canberra, Australia. Email: andrew.leigh@anu.edu.au. Website: andrew.leigh@anu.edu.au
[1] Data going back to 1979 at the national level are available from Australian Bureau of Statistics (ABS) publications, but earlier data must be purchased from ABS Consultancy Services.
[2] Chapman et al (2006) exclude data after 2003 from their analysis, based on concerns over data reliability. The number of deaths identified in each year differs somewhat between Chapman et al (2006) and Baker and McPhedran (2006). It is unclear why this is the case.
[3] ARIMA(1,1,1) refers to an auto-regressive integrated moving-average model, with integration of order 1 and first-order serial correlation and moving average components.
[4] Chapman et al do not estimate the statistical significance of this difference, but given the prior history of mass shootings in Australia, the probability that this change in the frequency of mass shootings was due to mere chance is well below 1%.
[5] In the case of the gun buy-back, this assumption is clearly violated. Reuter and Mouzos (2003:132) show that the number of firearms handed back in Victoria in 1997 was 4,300 per 100,000 people, a higher rate than for Australia as a whole (3,400 per 100,000 people).
[6] This is broadly the conclusion in their published paper. Statements to the media, however, were much stronger. In a summary of the research released at the Sporting Shooters’ Association of Australia website, the conclusions are:
• ‘The reforms did not affect rates of firearm homicide in Australia.
• The reforms could not be shown to alter rates of firearm suicide, because rates of suicide using other methods also began to decline in the late 1990s.
• …
• It must be concluded that the gun buyback and restrictive legislative changes had no influence on firearm homicide in Australia.
• The lack of effect of a massive buyback and associated legislative changes in the requirements for obtaining a firearm licence or legally possessing a firearm has significant implications for public and justice policy, not only for Australia, but internationally.’
Source: ‘Gun Laws and Sudden Death: Did the Australian Firearms Legislation of 1996 Make a Difference? Executive Summary’. Available online on 18 December 2007, from www.ic-wish.org/Executive%20Summary.pdf
[7] Koper and Roth are clearly aware of these difficulties, and highlight them in their paper, noting that ‘the law has not produced a clear impact on gun violence’ (2001a:69), although they suggest that the ban likely ‘contributed to a reduction in gun homicides’ (2001a:33).
[8] Although much of Kleck (2001) focuses on the relatively small impact that the 1994 US federal ban on assault weapons and large capacity magazines had on overall gun ownership (a critique that is not applicable to the Australian NFA), Kleck concludes by saying ‘there is no effective way to assess the impact on crime rates of a unique national policy change’ (2001: 80). This critique is potentially applicable to the Australian reform.
[9] An exception is Chapman et al (2006), who exclude data from 2004 that was considered to be of dubious quality. No results are reported in that paper from different time periods, however.
[10] We could not obtain the full set of data underlying Figures 1 and 2 in Baker and McPhedran (2006) from the corresponding author. Our results are therefore based upon Table 1 in Baker and McPhedran (2006), supplemented with data kindly provided by Jenny Mouzos of the Australian Institute of Criminology, and the most recent population data from the Australian Bureau of Statistics. We first used precisely the data laid out in Baker and McPhedran (2006) and estimated ARIMA(1,1,1) models for the period 1979 to 1996. We did not have access to the same statistical package used by Baker and McPhedran, and initially used STATA. STATA did not yield estimates similar to those reported by Baker and McPhedran. For instance, the firearm homicide model did not converge. Simply changing the population figures to more recent ABS estimates helped somewhat. Switching to the statistical package R brought our estimates considerably closer to those in Baker and McPhedran; the results reported here are therefore from R. We consider the estimates from R to be more reliable, but the sensitivity of the estimates to the statistical package used is somewhat concerning, and itself suggests that the ARIMA(1,1,1) specification is not ideal. Note, however, that regardless of the statistical package used, and despite coming quite close on firearm suicides, we could not replicate Baker and McPhedran’s predictions of the average firearm homicide rate after 1997 in the absence of the NFA. We also note that the statistical ‘test’ used by Baker and McPhedran is more heuristic than formal. McDowall and Loftin (2005) discuss more formal tests for structural breaks in an I(1) model. Here, we nonetheless use the methods described in Baker and McPhedran, in order to focus attention on the robustness of the results to modelling changes. All data used in this paper and the programs used to obtain the statistical results (in STATA and R), as well as details of those results in numerical and graphical form, are available at www.wlu.ca/sbe/cneill.
[11] McDowall (2002) notes that to date little attention has been paid to the possibility that crime rates can be described by non-stationary processes. Using a time series from 1925 to 2000, described in McDowall and Loftin (2005) as ‘short’, he finds that US homicide rates are well-described by an I(1) model with first-order serial correlation.
[12] Results available on request.
[13] In principle, a variety of statistical approaches could be used to model the effect of the NFA on firearm deaths. Exploring the full gamut of approaches may be a useful exercise, but it is beyond the scope of this paper.
[14] The term ‘replicate’ has been the subject of some recent controversy. Here, we use it in its most limited sense, to mean recovering the same estimates in a published paper using the same data set. We see such ‘replication’ as a first, rather than a final, step in assessing a paper’s conclusions.
[15] All data and statistical programs used in this paper are available at: www.wlu.ca/sbe/~cneill
[16] This is the concern that Britt et al (1996) have with the use of non-firearm deaths as a control for firearm deaths.