Step 7 – Calculation of the benchmark ranges
The process of establishment of benchmark ranges is described in the SBB document. 85. Having chosen ranges, as described in Step 6, the process for each range is as follows:
- Calculate arithmetic mean of ratios in each range.
- Having calculated the mean, calculate the absolute difference of each ratio from the mean within the range.
- Choose a range that represents 30% of the population within that range – the range is not necessarily symmetrical around the mean. As it minimises the aggregate absolute differences calculated in the preceding step, it will include the mean, but won’t necessarily have an equal number of observations above and below the mean. Round the top and bottom points (that is, ratios) of each range to the nearest percentage point. The rounding means the ranges can be greater or less than 30% of that range’s population.
Initially we replicated the published ratios for each of the 6 industries. We were able to re-produce the results as provided when adopting the same ranges as selected by the ATO analysts. These ratios are shown in Appendix A, below.
We also were able to check these ratios against those published on the Small Business Benchmarks pages of the ATO website.
As noted above, the published ranges are rounded to the nearest percentage, which can result in more or less than 30% of the population in the published range. For example, the medium range for the Total expenses ratio for Ice Cream retailing covers about 34% of the medium range population.
The decision to make the published range cover about 30% of a turnover range’s population is a policy decision, but we note that given it is a relatively narrow range, there are many observations that are outside the range.
It should be remembered that for a ratio such as Total Expenses to Turnover, entities with ratios below the mean are not a cause for concern unlike those with high values for this ratio. Entities with a low value are effectively declaring relatively high profit margins compared to their peers so are less likely to be in breach. It is the entities with higher Total Expenses to Turnover ratios that are of greater concern.
The graph on page 17 Review of statistical methodology used in producing small business benchmarks (PDF, 1.5MB)This link will download a file shows a plot of the individual Total Expenses to / Turnover ratios for the 'High' turnover range (greater than $600,000) for Ice Cream Retailing. The horizontal lines show the average, as well as the upper and lower bands for the published ratios (84% to 93%).
As the plot of the individual ratios for each entity show, the progression is not smooth and symmetrical. The published range captures the observations surrounding the mean, but as this range is intended to cover only 30% of the range population, there are many entities’ ratios outside the range. Concentrating on those entities with ratios above the top of the published range, there are 24 entities (out of a population of 53) with ratios above the range.
Ice Cream retailing is an industry with relatively low numbers of members; indeed the 'High' range population is only just above the minimum threshold (of 50) for publication for a key benchmark ratio (Step 8).
A similar graph may be derived for Beauty Services. As one of the selected 'medium-sized' industries, the 'High' range (more than $400,000 turnover) for Beauty Services contains 355 entities in the population for the Total Expenses to / Turnover ratio.
Nevertheless, it may be observed that there are 130 entities with Total Expenses to Turnover ratios higher than the top of the published range. This represents about 37% of the 'High' population for the Total Expenses to / Turnover ratio.
For a selected 'Large' industry, such as Plumbing Services, there are even greater numbers of entities within the population. For the 'High' range (more than $600,000 turnover) for Plumbing Services, there are 2,114 business entities.
The distribution of ratios as shown on page 19 Review of statistical methodology used in producing small business benchmarks (PDF, 1.5MB)This link will download a file appears to be a lot smoother than the previous graphs. This reflects the larger number of entities.
For Plumbing Services there are 810 entities (or about 38%) that have Total Expenses to Turnover ratios higher than the upper threshold of the published ratio.
It may be observed that the published range in each of the 3 graphs above is quite narrow, as it is selected to only encompass 30% of the observations within each range. The top and the bottom of the range are selected so the 30% range minimises the differences in absolute value from the average. In other words, rather than take 15% on each side of the average, the range is effectively shifted slightly so a flatter part of the distribution will form the range.
In general, the 'High' range for the Total Expenses to Turnover ratio exhibits a greater degree of variability that the 'Low' or 'Medium' range. The graph on page 20 Review of statistical methodology used in producing small business benchmarks (PDF, 1.5MB)This link will download a file This link will download a fileis adjusted to show the 3 ranges on the same axes for the 'Low', 'Medium' and 'High' ranges.
The 'Low' range ($65,000 to $200,000 turnover) contains the highest population. It may be observed that this range is relatively less skewed than the 'Medium' ($200,000 to $400,000 turnover) or 'Large' (over $400,000 turnover) ranges. The 'Low' range also has a wider range (68% to 82%) than the 'Medium' or 'Large' range.
The percentage of entities with Total Expenses to Turnover range higher than the top end of the published range are 38% for the 'Low' range, 41% for the 'Medium' range, and 37% for the 'High' range.
Our analysis has confirmed the correct calculation of averages and ranges for the Total Expenses to Turnover, and Cost of Sales to Turnover benchmarks; that is the calculations are in accordance with the description provided in the SBB document. Further the resulting benchmarks are the same as those published on the ATO website.
The decision to publish ranges (provided the tests of step 8 are met) that reflect a band containing 30% of the range population is a policy decision. However, this choice has implications, in that high proportions of entities exhibit ratios that are higher than the published bands.
In making a comparison between an individual business entity’s ratio and the published benchmark, relevant staff of the ATO must be aware of the large proportions of entities that are higher than the published ranges. If using the ratio (together with other measures) to select an entity as a candidate for audit, it would seem reasonable to only select entities with ratios significantly above the published ratios.
Clearly the use of benchmark ratios cannot be a sole indicator of breaches regarding the cash economy, but it is reasonable to expect that entities with a higher probability of breach would exhibit a higher Total Expenses to Turnover or Cost of Sales to Turnover ratio.
|
Beauty services – top of range |
Key benchmark ratio annual turnover range $65,000–$200,000 |
Key benchmark ratio annual turnover range $200,000–$400,000 |
Key benchmark ratio annual turnover range more than $400,000 |
|---|---|---|---|
|
Cost of sales to turnover published |
25% |
24% |
24% |
|
Only 5% of population higher |
41% |
37% |
35% |
|
Total expenses to turnover published |
82% |
87% |
92% |
|
Only 5% of population higher |
97% |
98% |
99% |
In statistical terms, the calculations of the ranges is valid, but the analysis above shows the relationship between the chosen ranges (broadly reflecting 30% of the population) and the subsequent use. In applying the ratios, particularly to identify potential audit targets, it would not be reasonable to choose a business entity with a ratio slightly above the published band.
The distribution and skewness of ratios within each turnover range determines the width of ranges that capture differing proportions of the population. The preceding table shows differing amounts between the top of the published bands and a band designed to exclude only the top 5% of individual ratios.
The ATO may consider the publication of other bands for internal use, particularly for audit targeting. These could be constructed on the basis of traffic lights, whereby bands could be calculated reflecting red, amber or green risk profiles.
From their past experience, the ATO may be able to reach an appropriate 'a priori' expectation of the proportion of businesses in an industry that shall require further investigation. This proportion could then be used as the basis to calculate ranges of ratios for ATO internal use. Such ranges could better formalise the identification of entities whose ratios are 'significantly outside the published ranges'.
Alternative range testing
As stated earlier, the establishment of turnover ranges (generally 'Low', 'Medium' and 'High') requires some judgment. In order to better understand the impact of the chosen ranges, we re-calculated the average and 'published' ratio ranges, after changing the turnover ranges.
The results are shown in Appendix A, following the Key benchmark ratios which were published, and confirmed in our calculation.
An analysis of these alternative calculations yields few surprises. The 'published' ratios vary from the actual published ratios as would be expected; a greater turnover range leads to a greater range of ratios in the 30% band surrounding the mean.
These alternative calculations confirm the view that while the statistical methodology and calculation is correct, the usage of the ratios is circular. Ratios should only be used for benchmarking the relevant industry and turnover range for which they have been derived.
Step 8 – Normality and homogeneity testing
The selection of key benchmark ratios and secondary benchmark ratios for publication is dependent on normality and homogeneity testing.
Given the earlier steps of the culling of the starting population, the exclusion of outliers, the further exclusion of entities with turnover from $30,000 to the bottom end of the 'Low' range, and the classification into 2 to 3 turnover ranges, the final groupings for analysis and potential publication have been significantly adjusted. Accordingly it would be surprising if a large number did not meet the normality and homogeneity tests, at least for those industries with larger populations.
It is those industries and turnover ranges with smaller populations that could fail these tests, leading to a decision not to publish the benchmark ratios.
Step 9 – Review of the benchmark ratio output – quality assurance and other testing
119. This step involves several tests to provide confidence of correct calculation, and accordance with expectation. The steps include:
- Comparison with previous year’s benchmarks, on the expectation there would not be significant variation in the space of a year.
- Size and spread across the benchmark range, whereby those ranges that are relatively wide are investigated further.
- Trends of the benchmark ratios across a turnover range. For most industries it would be reasonable to expect consistent patterns across the three turnover ranges (for example, middle range between low and high ranges). Any discrepancies are investigated.
These processes are sensible, and another indicator of the thorough approach taken by the ATO in the derivation of the Benchmarks.
Step 10 – Preparation of industry names and overviews
This step is primarily a policy decision without statistical methodology implications.
Step 11 – Preparation of documents for publication
This step is primarily a policy decision without statistical methodology implications.
Step 12 – Small business benchmark publication approval
This step is primarily a policy decision without statistical methodology implications.