Methods and tools to evaluate the quality of vital statistics
Using tabulation and analysis to check the data
When microdata are aggregated and cross-tabulated in different ways by varying characteristics, it is possible to see errors that have not been visible before. For instance, when looking at the data over time, or comparing results with those in neighbouring districts, many errors may become visible when performing plausibility checks. This more analytical validation of the data is sometimes missing, and results in the tion of incorrect and inconsistent data. This can be avoided by following some SOPS in each data investigation phase (see figure below).
The simplest way to avoid this is to run a series of cross-tabulations covering all the different variables, or comparing observed values with those forecast or expected and checking major deviations before publishing. In addition to the numeric checking of the aggregated data that may be built into the tabulation process, an additional analysis and evaluation of the data are also required. When possible, charts and maps can be used to display results and check for outlying values for a better overview of large datasets. To avoid errors in large datasets, the experience from the best statistical agencies is that analysing the data internally is the best way of eliminating errors. Often, the implausible values discovered at the generation stage are a result of data compilation errors or misunderstood procedures.
Some data quality verifications can be performed by:
- Producing a set of verification tables that consist of basic tabulations for the majority of variables in the database by province or territory of occurrence;
- Sending verification tables to each provincial/territorial registrar of regional statistical offices for their review of the vital statistics, to ensure that their registry obtained the same results;
- Checking for internal consistencies – for example, by running frequencies and looking for outliers on certain data elements: How many centenarians are there? How does that compare to other countries or regions? and
- Comparing the most recent data year with past data years to detect any unusual or unexpected changes.
United Nations Department of Economic and Social Affairs Statistics Division (2001). Principles and recommendations for a vital statistics system, revision 2, UNDESA, New York.
Australian Bureau of Statistics (2012). Suicides, Australia 2010, ABS, Canberra.
Statistics Canada (2017). Motor vehicle accidents causing death, by sex and age group, Statistics Canada, Ottawa.