Effects of implementation support on children's reading outcomes following an online early reading programme: A cluster-randomised controlled trial

Well-designed computer or app-based instruction has a number of potential benefits (eg increasing accessibility and feasibility of high-quality instruction, reducing time and resources required for training expert delivery, saving instructional time). However, variation in implementation can still affect outcomes when using educational technology. Research generally suggests that without follow-up support after training, implementation of educational interventions is often poor and outcomes reduced. However, the extent to which this is the case when the core element of an intervention is computer or app-delivered is not yet clear. This study investigated the effects of providing ongoing implementation support for Headsprout Early Reading (HER, an early reading programme accessible via a computer or an app), to determine whether such support leads to better outcomes. Twenty-two primary schools (269 learners) participated in a cluster-randomised controlled trial. Eleven schools received initial training followed by ongoing support across the school year, whereas the other 11 schools received initial training and technical support only. Pre- and post-measures of reading skills were conducted using the York Assessment of Reading for Comprehension. We found no effect of implementation support on outcomes, and no effect of implementation support on delivery of the core element of HER. However, there were some effects of implementation support on the implementation of other HER elements relating to the responsiveness of educators to learners' learning within HER. These findings have implications for providing access to high quality online instruction in early reading skills at scale, with minimal training. More broadly, the current study suggests that well-designed computer or app-based instruction can yield positive outcomes with minimal implementation support and training. However, further research is required to ensure the interplay between learners' app-based learning and teacher intervention functions as intended to provide additional support for those who need it.

Practitioner notes

What is already known about this topic

• Well-designed computer or app-based instruction has a number of potential benefits (eg increasing accessibility and feasibility of high-quality instruction, reducing time and resources required for training expert delivery, saving instructional time).
• Implementation can still affect outcomes when using educational technology, and without follow-up support after training, implementation of educational interventions is often poor and outcomes reduced.
• The extent to which this is the case when the core element of an intervention is computer or app-delivered is not yet clear.

What this paper adds

• We found that providing implementation support for teachers and teaching assistants delivering Headsprout Early Reading (HER; an early reading programme accessible via a computer or an app) did not affect the reading outcomes of learners.
• We also found the implementation support did not affect delivery of the core, app-delivered element of the programme.
• However, there were notable differences in implementation of other aspects of the programme, particularly in relation to the role of the teacher or educational practitioner in managing the interplay between the app-based learning and teacher intervention for learners who require further support.

Implications for practice and policy

• These findings have implications for providing access to high quality instruction in early reading skills at scale, with minimal training.
• More broadly, the current study suggests that well-designed computer or app-based instruction can yield positive outcomes with minimal implementation support and training.
• However, the findings of this study identify some potential risk of an over-reliance on technology to facilitate the learning of all learners accessing the programme.
• Further research is required to ensure the interplay between learners' app-based learning and teacher intervention functions as intended to provide additional support for those who need it.