Abstract

One important goal in educational research is assessing the change of knowledge between two points in time since it allows to explain change in terms of cognitive variables, treatment effects and treatment interactions. Change of knowledge is usually assessed by using “gain scores” i.e. the difference between posttest and pretest measures. Although the unsuitability of gain scores has long been discussed (Cronbach & Furby, 1970), they are still nowadays employed and even preferred over more suitable alternatives due to its simplicity: two separate measures are transformed into an unidimensional score which is interpretable as gains of knowledge and usable in subsequent analysis. Issues of gain scores are that they (1) have low reliability, (2) cannot control for the initial status of knowledge—they correlate negatively with pretest scores, and (3) important cognitive variables such as intelligence and memory influence them only sparingly. Hence, gain scores are not an adequate estimator for the change of knowledge. In this presentation, a simple statistical model for the change of knowledge is developed in the setting of an educational intervention. The model provides a theoretical rationale to the criticisms of gain scores mentioned above. In the light of these results, this work proposes a new estimator for change that has the appealing features of gain scores but none of its drawbacks. Classical test theory and Taylor series expansions are used to estimate its theoretical reliability. Further work along this line might path the way to develop novel statistical tools useful for analysing educational data.