Impact evaluations for Government agencies

In 2004 Timmus was asked to run a national research project to investigate the effects of use of A-level e-learning materials (called SCHOLAR) on student attainment and motivation. This project ran for two years across three counties and within 56 schools.


The amount of data involved in this project was so epic that we had to buy a server and hide it in the under-stair cupboard! We visited a number of the schools in order to interview students and identify how they interacted with the e-learning. Tabetha had to learn enough MySQL to amalgamate data from the e-learning software tool with attainment data from each region. This project involved planning a robust experimental design, as well as lots of data management, cleansing, testing and coding to ensure the data were reliable. Only then could the analysis begin!


The first year report (2005) found that a minority of students used the e-learning a lot, and these students did do better than anticipated in their A-level exams. Most notably, evidence was found for a correlation between higher e-learning use and higher student attainment at A2 in comparison with AS. Due to the interesting nature of these results it was decided to continue research into a second ‘Phase 2’ year. No such correlation could be found in the second year, and the second year report (2007) investigated a number of issues relating to the influence of academic motivation on both attainment and the use of e-learning materials.