Reliability assessment

How we assess tax gaps for reliability

All gap estimates are assessed for reliability against 10 criteria. The reliability rating provides a transparent assessment of our gap estimates, drawing on the International Monetary Fund (IMF) evaluation framework and our expertise. We summarise this in a rating assessment for each gap estimate.

Reliability criteria

The 10 reliability criteria are considered of equal importance.

We sort them into 3 groups by evaluation stage.

Stage 1 – Evaluation of the estimation framework

1. capture the appropriate tax base
2. cover all potential taxpayers
3. account for all potential forms of non-compliance
4. no overlap within or between any components of the framework.

Stage 2 – Evaluation of the methodology

5. evaluate the approach used against the assessment criteria for that methodology
6. ensure the most appropriate method is used and results are validated against supporting information
7. sensitivity to the underlying model, assumption and structure
8. assessment of assumptions, judgment or expertise.

Stage 3 – Evaluation of the internal process and delivery

9. evaluate the quality of the management process
10. the analysis provides insights into the drivers of a gap estimate.

Reliability ratings

For each estimate, each reliability criterion is scored from zero ('poor or missing') to 30 ('excellent'). The sum of these scores determines the reliability rating.

The total reliability score ranges from zero to 30 and is placed into one of 5 categories.

• Very low is a score of zero to 10 – the results are preliminary or interim in nature, often being a pilot estimate in its first years of production. The estimate
  • may have several issues that compromise its reliability
  • may have a large or unknown margin of error
  • is not confirmed by other independent analysis
  • results provide very little information and could be misleading.
• Low is a score of 11 to 15 – many factors are not considered in the estimate. The estimate
  • has a material margin of error
  • may be partially confirmed by other analyses
  • should not be used to provide insight into population compliance
  • may provide direction for further research and analysis
  • improvements, when made, may significantly alter the gap estimate.
• Medium is a score of 16 to 20 – several factors are not considered which, if addressed, may change gap estimates by a limited or immaterial degree. The estimate
  • has an acceptable margin of error
  • is derived from an appropriate calculation methodology
  • is materially confirmed by other analyses, such as risk models and intelligence scans
  • with caution and contextualisation, can provide insight into population compliance.
• High is a score of 21 to 25 – a small number of factors are not considered which, if addressed, may change the gap estimate to a very limited or immaterial degree. The estimate
  • has a low margin of error
  • is derived from a highly appropriate calculation methodology
  • is materially confirmed by other analyses such as risk models and intelligence scans
  • with caution and contextualisation, can provide insight into population compliance.
• Very high is a score of 26 to 30 – all factors are considered. The estimate has a very low margin of error. The estimate
  • is derived from the most appropriate calculation methodology
  • is confirmed by other analyses, such as risk models and intelligence scans
  • can provide highly detailed insights into the levels of compliance across the population.