Data quality in demographic surveys: tests and experiments

The population of Nagaland, a state in North East India, decreased during 2001-2011, after growing at abnormally high rates during the past few decades. This is the first time after 1941-1951 that a state in India has witnessed an absolute decline in population. In light of this, the paper examines the census population estimates for internal consistency and also attempts to validate them using information on birth and death rates from other demographic surveys and information on gross school enrolment and electorate. The paper also checks if illegal immigration and politically-motivated manipulation could explain the discrepancies. The paper shows that the Census substantially overestimated population during 1981-2001. While the discrepancy in the 1991 Census could possibly be accounted for mostly by illegal immigrants from two neighbouring countries, a substantial portion of the discrepancy in the 2001 Census could possibly be attributed to deliberate inflation of population figures aimed at avoiding potential loss of seats due to the impending delimitation of electoral constituencies. Since the Census and the surveys that use it as a sampling frame are the bedrock of development planning in India, the findings question the Indian state’s institutional capacity to design empirically-informed policies.

This paper uses the newest census data, the 6th census of China in 2010, to check the data quality of the death. Analysis about the 4th and the 5th census of population in China shows that the death usually suffers obvious heap in some specific ages. Compared with the precious two censuses, this tendency in the 6th census decreases a lot, though still exists. Focusing on the infant mortality indicator, it is only about 3.8‰ computed by using data from the 6th census, which is almost impossible for present China. Based on the result and the infant mortality revealed by the National Ministry of Health in 2010, the 6th census omits about 140 thousands, even more than the registered number of death which is just 60 thousands. What’s more, with data of age-specific death rate of respective sex from the 6th census, the life expectancy at birth in 2010 of China can be computed using the life table method. However, the results are obviously too high, more than about 4 years old to the data from the United Nation, which means the death number suffers omission not only in the 0 years old but also in others. In addition, the omission has difference depending on when the death occurred.

Kenya, like most developing countries lack reliable vital registration. Knowledge of maternal mortality at a national level depends entirely on DHS surveys that provide information on the number of sisters who died during pregnancy, childbirth and postpartum. While such information remains valuable to assess levels and trends of maternal mortality, incompleteness of siblings’ history data and under-reporting of maternal deaths can seriously distort the estimated result. DHS data quality in estimating levels and trends of Adult mortality in Kenya using siblings’ histories has been documented elsewhere (see Masquelier, 2010). However, little work has been done to study the quality of DHS data in estimating levels and trends of maternal mortality in Kenya.

This paper aims to fill this gap by analyzing the evolution of DHS data quality in estimating levels and trends of maternal mortality in Kenya using the last three KDHS (1998, 2003, 2008-09). We focus on three major aspects: (1) completeness of data for the event; (2) under-reporting and misclassification of the events; (3) the plausibility of the patterns of maternal mortality indicators. We plan to combine the three surveys in order to estimate maternal mortality indicators (Maternal mortality rates, ratios and lifetime risk) between 1990 and 2008.

National statistics on maternal mortality vary considerably from international estimates: national figures for 2010 showed 5,670 maternal deaths in Latin America, substantially fewer than the 7,400 estimated by UNMMEIG or the 8,263 estimated by IHME. In Latin America, social and geographical inequality results in uneven coverage and quality of vital registration, and insufficient metadata to assess reliability. To facilitate an understanding of the methods, and to assist countries in the conciliation of national estimations with external sources, CELADE/ECLAC drafted a document comparing MMEIG, IHME and country estimates; explaining the approaches for measuring maternal mortality; and shedding light regarding interpretation of the multiple estimation sources.
Further, CELADE is working with eight pilot countries to explore data quality, assessments and adjustments. This paper reports findings of the pilot study, including a review on the coverage and quality of vital statistics; the state of the art of maternal mortality statistics at the national level; and an examination of discrepancies between sources, in the light of available metadata. Conclusions will be discussed with country stakeholders, resulting in recommendations on coordination, harmonization of figures and improvement of maternal mortality measurement.

This paper examines the reliability and stability of developmental idealism measures in Nepal. Developmental idealism is a cultural schema that contains beliefs and values favoring modern societies and families over traditional ones and that views modern families as causes and effects of modern societies. It also views the world as dynamic, with change from traditionality towards modernity. Earlier studies have shown that developmental idealism has been disseminated widely internationally, but provide little evidence concerning whether individual views of developmental idealism can be reliably measured or the extent to which such views are stable across time. We estimate reliability and stability of developmental idealism measures using panel data collected in Nepal. Our results indicate substantial levels of reliability, with levels equal to or nearly equal to reliability levels of standard value and belief items measured in general American surveys. There is also considerable stability of developmental idealism views across time.

In countries where the registration of deaths remains incomplete, the main sources of data on mortality are retrospective reports collected on the survival of close relatives. Birth and sibling histories are known to be plagued by recall errors but few methods exist to detect these errors. This paper introduces two simple approaches to assess the consistency between sibling histories and reports on the fertility of the previous generation. A comparison between the average size of sibships and the mean number of children ever born (CEB) to women of the previous generation indicates that a large proportion of siblings is omitted in Demographic and Health Surveys. On average the reported sibsizes are about 15 percent lower than expected from data on CEB. Discrepancies between reported and expected sibsizes are larger in sub-Saharan Africa than in other regions and they increase with the age of respondents. Second, a linkage of sibling histories reported by young women aged 15-18 with birth histories of their mothers when they live in the same household highlights frequent inconsistencies between mother's and daughter's reports on members of the same family. Daughters tend to report fewer siblings than expected from their mother's birth history, but more adult deaths.

EVALUATION OF QUALITIES OF AGE AND SEX DATA IN THE
2006 NIGERIA CENSUS AND 2008 NIGERIA DEMOGRAPHIC AND
HEALTH SURVEY
BY
Eleazar C. Nwogu and Hycinth C. Iwu
Department of Statistics,
Federal University of Technology,
P.M.B. 1526 Owerri
Abstract
Qualities of age and sex data in the 2006 census and 2008 Nigeria Demographic and
Health Survey (NDHS) have been investigated in this paper. Both graphical and algebraic
methods were applied to evaluate the data. Results of the analyses show that the age and
sex data in the 2008 NDHS are still characterised by pronounced preference for the end
digits 0 and 5 and avoidance of the end digits 1, 3, 7 and 9. Age shifting and age
exaggeration are also pronounced in both the surveys. However, there are indications that
data quality has improved substantially over the results from previous surveys in the
country. Therefore, a critical appraisal of the cause(s) of persistent poor data quality in
Nigeria, education of the populace, greater emphasis on registration of births, adequate
training, supervision and remuneration of enumerators, among others have been
recommended.
Key Words: Data quality, Myre’s index, Age-sex accuracy index, digit preference, age
shifting, age exaggeration.

Collecting data on sexual behavior based on recall has always been a concern. In this paper we compared data gathered from face to face interviews with daily information provided using the mobile phones through the Experience Sampling Method (ESM). With information from a purposive sample of 90 couples in Metro Cebu where one of the spouses/partners is a participant of the Cebu Longitudinal Health and Nutrition Survey, we determined whether the reported coital frequency differed according to the manner the data were gathered. We used the endline survey to capture the seven-day recall information and the daily responses of the past seven days (to mobile phones) to compare the consistencies of the responses. Results revealed that with respect to coital frequency, data from mobile phones were higher than face-to-face interviews with respect to sex age group, marital status, work status, education and household assets. There were differences also observed between couples with husbands/males reporting higher frequencies than their wives/partners irrespective of the method of data collection. The findings highlight the importance of data collection methods in gathering sensitive information in less developed societies where open discussions of sensitive issues like sexual activities remain taboo.

Problems with reproductive health data in non-Western settings are widely noted and have been the focus of innovative methodological work. Collection of improved data on contraceptive use and abortion generally involves methods for distancing the interviewer from the respondent through forms of self-administered questionnaires (ACASI, etc.). We discuss an alternative approach which breaks from the “stranger interviewer norm." We present data from a unique experimental design recently fielded in the Dominican Republic levels of familiarity linking interviewers and respondents are randomly varied and test how this variation alters responses. Initial evidence suggests that contraceptive use estimates obtained by more traditional, stranger interviewers in our sample resemble estimates obtained by the DHS. Estimates obtained by local interviewers however are far lower. Our findings suggest that levels of familiarity strongly affect how contraceptive use data are reported by respondents. Further analysis is focused on indirect methods to validate the widely varying reports.

When it comes to the topic of sex, analysts are often skeptical. And with good reason: respondents may lie or forget the juicy details of their intimate lives, and interviewers may exercise authority in how they capture it. In between the two lies a more fundamental problem endemic to social life: how people appear to others is never unmediated nor unfiltered. In this paper we use data from a cross-sectional HIV prevalence and sexual behavior survey conducted in 2010-2011 in a rural Southern African setting to explore the broader question of who says what to whom about their sexual lives. Preliminary results show a consistent age effect across outcomes-- that respondents report more “moral”, responsible sexual behavior to older fieldworkers; and a strong sex effect-- that men report more sexual partners to female fieldworkers. Understanding fieldworker effects on the production of sexual behavior survey data serves both methodological and theoretical goals.