B.
Qualitative interviews
In preparation for this article, the authors also contacted 40 experts in this field, including researchers, academicians, policy makers and practitioners, working both within the United States and Europe and in developing countries such as India, Nepal, Zambia and Afghanistan. Both users and developers of ICT solutions were contacted. Of these 40 individuals, 25 agreed to in-depth telephone interviews. This is a unique component not found in traditional desk studies. The goal of this effort was to request unpublished reports on the educational impacts of ICT in the developing world and to obtain a more detailed first-hand account of the challenges associated with developing, implementing and evaluating ICT interventions in developing countries. Interview questions depended upon the expertise of the individual, but generally included:
•
In your opinion what is the future of low-cost and other ICT devices in educational sector in developing country contexts?
•
Is there a widespread need for these types of solutions?
•
What changes should be made, if any, to the use and development of ICT in education?
•
What, in your opinion, are the two or three most important considerations when planning and deploying ICT for education?
•
What are the challenges to implementing these solutions in developing/emerging countries?
•
(For users of ICT solutions) What resources are available and what do you still need? What are the challenges to using these types of devices? What solutions are you most excited about?
In the full length version of this report, we present and discuss existing literature (divided into experimental evaluations and qualitative case studies) on the impacts of educational interventions with ICT components on student learning outcomes in developing countries. We divide the research into two categories: experimental evaluations and qualitative evaluations that use a case study approach. In this abridged version of the report, we summarize the challenges that exist to conducting evaluations of these types of interventions in the developing world and conclude with a review of the lessons learned from investment and activities witnessed during the past decade, including successes and failures, related to ICT use in education, and recommendations on how these lessons can inform policy dialogues and intervention development within the education sector in future years, focusing on the areas we believe are most salient for practitioners and policymakers.
IV.
Experimental Evaluations Assessing Impact
Few experimental evaluations have been conducted on the impact of ICTs on student outcomes in the developing world.ii Advocates frequently propose ICTs in the classroom, but often do so with little rigorous evidence to support their claims (F. Barrera, personal communication, August 25, 2008). Additionally, very few randomized controlled trials (RCTs) have been conducted and as a result, important questions remain unanswered, such as whether differential effects across subgroups exist, whether certain ICT interventions are more successful than others, and what factors mitigate and enhance the success of ICT interventions.
Studies employing rigorous methodologies, such as RCTs, allow researchers to generalize study findings beyond the study participants, an important design consideration when funding for evaluation is limited. By conducting experiments, researchers can attempt to construct cause-and-effect relationships and subsequently determine what types of interventions “work” and what do not. But even with the careful control of variables, causal relationships can be difficult to establish within complex social phenomena (Mertons, 2005). Ethical concerns can also surface when applying a