Cape Town Data Quality Workshop: Measurement of Development Indicators

 

In this blog Andrew Kerr, economist and a Senior Research Officer at DataFirst, University of Cape Town, reports back on a 2- day workshop held at the River Club in Cape Town on 6^th and 7^th July 2017 organised by Professor Martin Wittenberg, Director of DataFirst, with support from the UK Data Service.

The workshop consisted of several themes for the various sessions: Measurement of individual and household well-being, Access to household energy and household services, New forms of data and innovative approaches to measurement and Labour market data.

Attendees included academics from a variety of Universities (Cape Town, Pretoria, Stellenbosch) and a range of disciplines (Economists, Astronomers, Health and Energy researchers), as well as representatives from Statistics South Africa (StatsSA), the Department of Basic Education and the Office of Astronomy for Development (OAD) of the International Astronomical Union.

Focus on energy and new forms of data

The sessions on energy and new forms of data showcased some of the results from our ESRC-NRF funded international partnership between DataFirst and the UK Data Service, Smarter Household Energy Data.

Using sources of data outside the domain of traditional survey and administrative data is often seen as a challenge for social scientists, due to the unfamiliarity of and trust in the data. A lot of data preparation and manipulation has to go into preparing of data sources derived from real-time measurements. Takwanisa Machemedze, formerly of DataFirst, presented a paper on using nightlights data to measure rural electrification. He showed that the nightlights data tracked the roll-out of electricity connections in a health and demographic surveillance site in the east of the country.

Also using satellite data, Tawanda Chingozha, a PhD student from the University of Stellenbosch, presented a paper using satellite data to estimate changes in land under cultivation due to land reform policies in Zimbabwe. The conclusion was that the land acreage devoted to cultivated land decreased after the fast track land reform programme.

Martin Wittenberg presented a paper with Tom Harris on electricity connections and household formation, as featured in an earlier blogpost using the National Income Dynamics Study (NIDS) suggesting that aggregate electricity statistics, such as access rates, conceal a considerable degree of the complexity and volatility that is inherent in the development of electricity access. Instead he suggests that policy makers involved in electricity roll-out need to consider that household electricity access is a complex outcome of time-variant processes: net connections and household formation and dissolution processes.

Wiebke Toussaint from the UCT Energy Research Centre documented the Domestic Load Research Project, a yearly survey of several hundred Eskom customers, the household data that has been produced by the project and the ERC’s aims to make some of the data available to the public. Grant Smith and Kathryn McDermott from the UCT School of Economics and JPAL presented a paper that showed the effects of changing to prepaid electricity meters as well as giving insights into the difficulties of using administrative data, in this case from the City of Cape Town.

Kathryn McDermott from the UCT School of Economics and JPAL speaking

Chris Park from the UK Data Service submitted a presentation on behalf of Simon Elam from UCL, on “Data Quality: The elephant in the (big data) room”, which looked at the opportunities for using and issues with the quality of data from smart meters.

Measurement of individual and household well-being

Nilmini Herath from JPAL Africa at UCT presented a paper sharing JPAL’s insights gained in running surveys in South Africa. The presentation led to a helpful cross-pollination of ideas with the Stats SA participants who were interested in understanding and improving fieldwork quality in Stats SA data. Emmanuel Bakirdjian of JPAL Africa at UCT presented a related paper giving insights into how different forms of measurement can complement or improve the self-reported data traditionally used in household surveys.

Two of the presenters were students on the first (2016) cohort of the new Post-Graduate Diploma in Survey Data Analysis for Development at UCT, which has been put together by DataFirst, together with SALDRU and the School of Economics. Karabo Sebolai from StatsSA presented a paper comparing imputation and reweighting adjustments for non-response in the Agricultural Survey run by Stats SA, whilst Phumudzo Madzivhandila, also from Stats SA, examined the robustness of multidimensional poverty estimates to different weights for the various components of the poverty estimates.

Professor Steve Koch from the University of Pretoria presented a paper estimating equivalence scales for South Africa, using the 2010 Income and Expenditure Survey. One implication of his work is that standard procedures used for adjusting incomes for household size and composition (normally done by calculating per capita figures) probably over-adjust. This would make big households look poorer than they probably are.

Martin Wittenberg presented work which suggested that there were sampling issues with the Living Conditions Survey. It looked as though the survey was finding fewer rich people at the end of the survey period than at the beginning. This could be related to fieldwork fatigue. Similar questions have been raised about the diary method of collecting consumption data over a whole year.

Measuring labour market changes

In the final session, Andrew Kerr from DataFirst presented two papers using the PALMS data (Post-Apartheid Labour Market Survey series), a compilation of Stats SA labour market data from 1994-2015. The first was on how the standard errors in the Quarterly Labour Force Survey (QLFS) are estimated by Stats SA. It suggested that quarter and quarter changes are probably more noisy than consumers of the data think. The second dealt with the quality of the earnings data in the QLFS and the changes over the period that make comparability over time more difficult.

Andrew Kerr from DataFirst speaking

There was lively discussion on all the papers which continued over lunch, tea and the conference dinner. Fieldwork quality, measurement error and how to share different types of data came up repeatedly. There was also very useful cross-disciplinary engagement, with the Office of Astronomy for Development particularly interested in the use of remote sensing for measurement.

Louise Corti , Principal Investigator on the UK side of the UK-South Africa project notes that “The International Astronomical Union (IAU) is the largest body of professional astronomers in the world who have set up the Office of Astronomy for Development (OAD), in partnership with the South African National Research Foundation (NRF). This is a wonderful opportunity for collaboration across disciplines to harness powerful data resources in the investigation of critical development issues”.