Richardson, Grant; Smith, David --- "The Readability of Australia's Goods and Services Tax Legislation: An Empirical Investigation" [2002] FedLawRw 17; (2002) 30(3) Federal Law Review 475

The sum total (including interactions) of all those elements within a given piece of printed material that affects the success which a group of readers have with it. The success is the extent to which they understand it, read it at optimum speed, and find it interesting.^[22]

It can be seen then, that Dale and Chall identified three aspects of the reading process: comprehension, fluency (reading speed), and interest.^[23] We submitt here that both comprehension and fluency are important considerations in the readability of taxation law; though perhaps 'interest' is less so. For this reason, a definition or measure of readability which does not focus on reader interest may be considered more appropriate in this particular circumstance.

In keeping with this expostulation, and in order to be consistent with Smith and Richardson,^[24] the Flesch Readability Index (or Flesch Reading Ease Index) is used in this study to measure the nature of readability. Flesch also acknowledged the importance of reader interest in any analysis of readability and, as a consequence, developed the Flesch Human Interest Index.^[25] The Human Interest Index has been largely ignored in subsequent research and is therefore not utilized here.

As Tan and Tower have suggested, there appears to be general agreement in the taxation literature that a low level of readability is one of the most important factors involved in taxation complexity.^[26] Cooper has identified 'clarity of expression' as being one of the most important issues referred to by taxation commentators when discussing the simplicity (or complexity) of a taxation system.^[27] It is assumed, therefore, that readability (which emphasises comprehension and fluency) is an important consideration in attempts to simplify, and thereby improve, taxation law, with perhaps the other main consideration being simplification of taxation policy.

IV RESEARCH METHODOLOGY

The use of readability formulae is the most commonly used method in contemporary research for assessing readability.^[28] Moreover, the use of readability statistics to measure simplicity of the wording of taxation legislation is supported by its prior use in taxation research.^[29] Readability indices are said to 'take into account characteristics of writing style that are measurable and then evaluate the extent to which each identifiable attribute impacts on reading difficulty.'^[30]

Readability statistics do not necessarily consider conceptual difficulty, semantics, reader characteristics and presentation of the material (such as font size, layout of text, graphics and tables). Consequently, they cannot result in an absolute measure of clarity.^[31] However, prior research has shown that readability indices can be used to assist in predicting the readability of business and legal documents. Indeed, the United States Internal Revenue Service has used the Flesch Readability Index to measure the readability of taxation forms and instruction booklets many times in the past.^[32]

There is also a plentiful amount of research in the accounting literature: at least 25 studies over the past 40 years (including some in Australia, Canada, Hong Kong, the United Kingdom and the United States) that have investigated the readability of narratives in company annual financial reports. The Flesch Readability Index was the dominant choice of researchers as the measure of readability in these studies.^[33]

Specifically, the Flesch Readability Index gives written documents a reading ease score from 0 (most difficult to read) to 100 (easiest to read).^[34] A score of between 60 to 70 is considered to be acceptable. Furthermore, the Flesch Readability Index is the most widely used readability formula outside education circles.^[35] It has been extensively validated in the past and found to be highly reliable.^[36] Finally, studies have also shown high statistical correlations between Flesch formula scores, and scores obtained using other readability indices.^[37] The index is calculated using the following formula:

Reading Ease Score = 206.835 – 0.846wl – 1.015sl

where:

• wl = number of syllables per 100 words; and
• sl = average sentence length in words.

This calculation can be undertaken using a computer program (for example, Microsoft Word 97). According to Harrison, computers are usually quite accurate in counting syllables, with the few errors made tending to be in either direction, thus cancelling each other out.^[38] By comparison, human calculation has been shown to lead to highly unreliable results. Nevertheless, Richardson and Sawyer have noted that some computer programs appear to have difficulty in processing large sentences.^[39] To this end, large sentences were checked manually in this study and results were adjusted if they were found to be inappropriate.^[40]

Given the size of the current study and the fact that computer calculation has been demonstrated to be considerably more accurate than human calculation, a computer program (viz. Microsoft Word 97) was used. Owing to the large number of the GST sections analysed, it was felt that the possibility of error affecting the results would be insignificant. In keeping with Harrison's assertion,^[41] it was assumed that such errors were likely to cancel out across the entire sample. Furthermore, many annual report readability studies published in reputable accounting journals have used computer programs to calculate Flesch scores.^[42]

Consistent with the research methodology applied by Tan and Tower,^[43] Richardson and Sawyer,^[44] Woellner et al^[45] and Smith and Richardson,^[46] the current study uses the Flesch Readability Index to measure the simplicity of a random sample of sections from the GST Act. These results are then compared to results obtained by Smith and Richardson who applied a similar methodology to ITAA36 and ITAA97.^[47]

The sample framework excluded sections from the GST Act of less than fifty words in keeping with Harrison's assertion that fifty words is generally taken to be a length below which text samples are considered inappropriate for analysis.^[48] Moreover, in keeping with Smith and Richardson,^[49] this study also employs the Flesch-Kincaid Grade Level (F-KGL) Index to measure reliability.^[50] The F-KGL Index rates text on a US grade school level and the formula is expressed as follows:

F-KGL = 0.39 (words/sentence) + 11.8 (syllables/word) – 15.59

While the authors acknowledge that this test is not ideal for testing the readability of taxation legislation, it does provide some further information about a text's readability and has therefore also been applied in this study. A score of eight on this test indicates that an eighth-grade US student can understand the document. Grade level indices such as the F-KGL Index have been demonstrated to be accurate to plus or minus one grade level.^[51]

V RESULTS AND DISCUSSION

As was previously mentioned in Section III, a document needs to achieve a score of between 60 and 70 (or better) on the Flesch Readability Index in order to be said to be sufficiently readable. The average Flesch scores shown in Table 1^[52] below indicate that the sections of the GST Act analysed have an average Flesch score of 40.3. This finding indicates low readability for the GST Act.

Table 1

Summary of the Readability of Australia's Tax Legislation

* Flesch = the Flesch Readability Index.

** F-KGL = the Flesch-Kincaid Grade Level Index.

*** PS (%) = percentage of passive sentences.

**** WPS (Av.) = average words per sentence.

Certain fundamental divisions and sections of the GST legislation stand out as worthy of special mention. For example, Division 2 of the GST legislation, which provides an overview of the GST legislation, has a division average of 59.8 (see Appendix) which, when rounded to the nearest whole number, meets an acceptable level of readability.

Important areas of the GST legislation are those that deal with compliance requirements, such as Divisions 31, 33 and 35, which deal with GST returns, payments of GST and GST refunds. These divisions had average Flesch scores of 35.8, 41.3, and 31.1 respectively (see Appendix). These scores all fall well below the acceptable benchmark range of 60 to 70. This indicates that these key divisions of the GST legislation are very difficult to read and understand.

Another important area of the GST legislation is Division 40 that deals with input taxed supplies. In particular, Sub-division 40A, which encompasses financial supplies, had a Flesch score of 46.9 (see Appendix). This score falls well below the acceptable benchmark range of 60 to 70 and demonstrates that this important Sub-division of the GST legislation is very complex in nature; a finding that is consistent with recent taxation literature dealing with this subject.^[53]

In their analysis of the readability of ITAA97, Smith and Richardson singled out Chapter 6 of ITAA97, viz. the Dictionary, as being worthy of special mention, since '[I]t would be expected that taxation legislation defining particular terms for taxpayers and therefore, in a sense, underpinning the entire Act should fittingly, score high on readability'.^[54] Smith and Richardson found a Flesch score for the ITAA97 Dictionary of just 29.9.^[55] Applying the same test to the Dictionary of the GST legislation revealed a very similar result, with a Flesch score of 30.3 being obtained (see Appendix). Given Smith and Richardson's suggestion that it is very important for taxpayers to be able to understand key terminology,^[56] these appear to be unsatisfactory results.

The results of this study purvey an interesting comparison with those obtained by Smith and Richardson concerning ITAA36 and ITAA97.^[57] This particular study investigated the readability of 95 sections from ITAA97 and the sections from ITAA36 that they replaced, and obtained an average Flesch score of 46.42 for ITAA97, and 38.44 for ITAA36. The results of this study concerning the GST suggest that, on average, the GST legislation is less readable than ITAA97. Nevertheless, the results for all three pieces of taxation legislation fall well below acceptable readability benchmarks.

Results obtained from using the Flesch-Kincaid Grade Level Index to measure readability provide a similar outcome.^[58] The average F-KGL Index obtained for the GST legislation was 11.0 compared to 10.76 for ITAA97 and 10.79 for ITAA36 (see Table 1).^[59] When these results are rounded to the nearest grade level, obviously, they are the same. Furthermore, an F-KGL Index of 11 (out of a maximum result of 12) provides additional evidence of the low readability of the GST legislation.

Of major concern is the number of GST sections that recorded the maximum possible F-KGL Index score of 12 (see Appendix). In particular, 205 sections out of 352 (viz. 58.2%) achieved the maximum F-KGL Index score of 12, indicating low readability for these GST sections. Indeed, the average F-KGL Index scores may have been even higher if the scale had extended beyond the threshold of 12.

A further important issue that should be considered with respect to the readability of the GST legislation is the use of passive sentences, notwithstanding that they are not directly incorporated in the readability formulae. Specifically, Table 1 shows that the use of passive sentences in the GST legislation is slightly lower than in ITAA97 (viz. an average of 20% in the GST legislation vis-à-vis 20.81% in ITAA97). It should be noted that on the basis of these results at least, the use of passive sentences in both the GST legislation and ITAA97 is not of a level that should be considered excessive.

The number of words per sentence can also affect readability of text. Of the GST sections analysed, average words per sentence (WPS) in the GST legislation was 20.9. This compares favourably with the results obtained by Smith and Richardson of 22.27 for ITAA97, and 39.02 in ITAA36 (see Table 1).^[60] Although the number of words per sentence is incorporated into the readability formulae, it is advantageous to isolate in order to provide additional information about a text's readability. Clearly, a reduction in WPS is desirable, as it can serve to improve readability.^[61]

A summary table of Flesch readability scores based on similar tables provided in Tan and Tower,^[62] Richardson and Sawyer^[63] and Smith and Richardson^[64] is presented below in Table 2. Table 2 categorises each section according to its Flesch score.

Table 2

Summary of Flesch Readability Scores

* The grade levels referred to are New Zealand grade levels, which generally correspond to Australian grade levels.

As can be gleaned from examining Table 2, the number of GST sections found to have Flesch scores under 30 was 91 (or 25.85%) of the total sample. Moreover, another 46.31% of the GST sections analysed had Flesch scores between 30 and 50. Consequently, Table 2 shows that 72.16% of the sections in the GST legislation can be regarded as difficult or very difficult to read, as a university education is required in order to understand these GST sections. In fact, only 11.93% of sections in the GST legislation met the acceptable readability benchmark of 60 or above and a Form 4 or lower level of formal education. These results show that the vast majority of the GST legislation is too complex to be understood by the average Australian taxpayer.

VI CONCLUSION

While the introduction of the GST in Australia did not fall under the auspices of the TLIP, it could be argued that efforts made to simplify any form of taxation legislation, including the GST, could be considered to be a desirable means of helping to reduce business taxpayer compliance costs.^[65] Therefore, the primary objective of this study was to test empirically the readability of the GST legislation in Australia.

This objective was accomplished by using empirical measures of the level of readability to operationalize Adam Smith's maxim of simplicity.^[66] A copious sample of sections was taken from the GST legislation and was subsequently tested empirically for readability levels using the Flesch Readability Index and the Flesch-Kincaid Grade Level Index. In addition, the samples were also analysed according to the use of passive sentences and average words per sentence.

According to the Flesch Readability Index, it was found that the sample sections of the GST legislation had an average Flesch score of 40.3. As this score falls well below the acceptable benchmark range of 60 to 70, this indicates that the GST legislation is in general, very difficult to read and understand. Moreover, more than 70% of the GST sections analysed had Flesch scores that show that they can only be read and understood by those persons who have at least reached a university level of education.

Results obtained from using the F-KGL Index when applied to the GST legislation were also found to be disappointing. The average F-KGL Index obtained for the GST legislation was 11.0. Moreover, an F-KGL Index of 11 (out of a maximum result of 12) provides further evidence of the low readability of the GST legislation. Of major concern was the number of the sections analysed that recorded the maximum possible F-KGL Index score of 12: viz. 205 sections out of 385 (or 58.2%). Indeed, the average F-KGL Index scores may have been higher if the scale had been extended beyond its upper bound of 12.

The use of passive sentences in the GST legislation was also considered as an important part of the study. The results indicated that the use of passive sentences was on average 20% in the GST legislation (see Table 1). However, the use of passive sentences in the GST legislation is not at a level that should be considered inordinate.

The number of words per sentence in the GST legislation was also investigated in the study. The average words per sentence in the GST legislation was found to be 20.9. Even though the number of words per sentence is incorporated into the readability formulae, it is useful to isolate this in order to supply additional information regarding a text's readability. Certainly, a reduction in WPS is desirable, as it can serve to improve readability.

The results of this study furnish a serviceable comparison with those obtained by Smith and Richardson concerning the readability of ITAA36 and ITAA97.^[67] Specifically, Smith and Richardson analysed the readability of 95 sections from ITAA97 and the sections from ITAA36 that they replaced and obtained an average Flesch score of 46.42 for ITAA97 and 38.44 for ITAA36 (see Table 1).^[68] The results of this study concerning the GST suggest that, on average, the GST legislation is less readable than ITAA97. However, the results for all three pieces of taxation legislation fall well below acceptable readability benchmarks.

Results obtained from using the Flesch-Kincaid Grade Level Index to measure readability provided a similar outcome. In particular, the average F-KGL Index obtained for the GST legislation was 11.0 vis-à-vis 10.76 for ITAA97 and 10.79 for ITAA36 (see Table 1). When these results are rounded to the nearest grade level, clearly, they are identical. Also, an F-KGL Index of 11 provides further evidence of the low readability of the GST legislation.

In addition, the use of passive sentences in the GST legislation is slightly lower than in ITAA97 (an average of 20% for the GST legislation vis-à-vis 20.81% in ITAA97) (see Table 1). Finally, the GST sections analysed showed that the average words per sentence was 20.9. This compares fairly well with the results achieved by Smith and Richardson of 22.27 for ITAA97 and 39.02 in ITAA36 (see Table 1).^[69]

The study's overall finding that the GST Act is very difficult to read and understand confirms Cooper and Vann's erstwhile proposition that 'the GST is not inherently simple, in contrast to the inherently complex income tax'.^[70] Likewise, when a general comparison is made to the readability of the ITAA97, it appears that the GST legislation is less readable. This finding could have implications for business taxpayers in the form of potentially higher GST compliance costs.^[71] Nevertheless, it should be remembered that simplification of taxation law (including the GST) is multifaceted in nature.^[72] Hence, improving the readability of the text contained in GST legislation is merely one important aspect of enhancing simplicity in the taxation system.

We acknowledge the concerns held by those who believe that the readability formulae, such as the Flesch Readability Index and the F-KGL Index, may over-simplify research into readability of taxation documents. As suggested earlier, as readability formulae do not take into account, among other things, conceptual difficulty, semantics, reader characteristics and presentation of materials, they cannot provide a faultless measure of clarity.

However, future researchers in this area could utilise sophisticated psycholinguistic and/or sociolinguistic techniques such as Cloze Analysis, Multidimensional Scaling, Association Analysis and Classification Analysis^[73] in combination with more fundamental readability formula approaches to provide further support for their empirical results. Future research could also be directed at replicating the present study's research design with respect to the many and varied GSTR Rulings that have been issued by the ATO as an aftermath to the introduction of the GST.

Appendix—Readability of the GST Legislation

A New Tax System (Goods and Services Tax) Act 1999