Reviewing Rasch decisions

Back in the 60s, British statisticians vigorously rejected a new analytical technique developed by Danish mathematician Georg Rasch. In so doing, they condemned the UK to an analytical backwater, argues Peter Tymms.

Educational measurementIn the 1950s, Danish mathematician Georg Rasch was presented with children’s reading test data and asked to analyse it for the Danish Ministry of Social Affairs. Quite ignorant of the traditional way of conducting test analyses, Rasch went ahead and developed his own approach, his simple yet elegant solution producing some remarkable insights and marking a breakthrough in test data analysis.

Out with the old?

For much of the twentieth century, analysts had relied on an approach known as Classical Test Theory (CTT) to guide their thinking when looking at data from examinations and other kinds of assessment. But as far back as the 1920s, psychometricians had highlighted the technique’s shortcomings, and worried about problems that CTT sidestepped.

Rasch’s new technique provided a tool that addressed many of CTT’s failings. For example, he put question difficulty and the ability of those who took the test onto the same scale – a very useful association known as ‘conjoint measurement’, which allows us to judge just how difficult each question is for every testee.

Rasch measurement also allowed for the difficulty of a test to be separated from the ability of the test-takers so that, unlike CTT, if an assessment were given to one group of subjects and then to another, the difficulty of the questions would remain constant.

These characteristics – and others – led theorists to consider the fundamentals of ‘measurement’, and to dismiss the suggestion that simply adding up numbers on a test or questionnaire would suffice.

Rasch reaction

Rasch’s ideas were revolutionary, and when we have revolutions we expect sparks to fly and arguments to be fought.

When Bruce Choppin brought these concepts to England in the 1960s and proposed creating question banks with known difficulties, his approach was attacked with vigour. It was claimed, for example, that the context in which a question was presented was so important that questions could not be ascribed isolated difficulties, and that pure single construct scales were not possible in education.  The traditionalists won the battle, and such was the fallout that Rasch was banished from the National Foundation for Education Research where it had seen a brief flourishing.

This stark reaction to the technique ‘condemned Britain to a 60 year regression’, according to Mike Linacre, a leading proponent of Rasch methodology. Rasch was not mentioned in any article in the British Educational Research Journal from 1980 until 2010. Fortunately, the ideas were developed and taken up by far-sighted workers elsewhere around the world, key amongst them Mike Linacre and Geoff Masters, then of the University of Chicago, and David Andrich of the University of Western Australia.

Rasch today

The chart below shows how often the phrase ‘Rasch measurement’ has been used in academic articles over five decades according to Google Scholar. The growth is dramatic, and in 2012 the phrase appeared 853 times – averaging out at just over two published articles per day!

'Rasch measurement' in Google Scholar

The UK Rasch User Group, established in 2006, is part of a drive to reinvigorate a UK movement which was started in the 1960s and then essentially suppressed for decades. Rasch analysis has been carried forward by key players who have done much to resolve crucial issues and refine the technique. It has had a major impact on international country-wide tests such as PISA, on computer adaptive testing, and in the thinking of psychometrians. We are still at a crossroads – but after decades in the doldrums, Rasch is finally beginning to gather momentum and fulfil its potential.

Peter Tymms is Head of the School of Education at Durham University and is one of the organisers of the annual UK Rasch User Group Day. This year’s event is sponsored by CERP and will take place on 8 March at Manchester Conference Centre.

References: 
  1. Andrich, D. (2004). Controversy and the Rasch Model: A Characteristic of Incompatible Paradigms? Medical Care, 42(1 suppl), I-7 to I-16.
  2. Bond, T. G., & Fox, C. M. (2007). Applying the Rasch Model: Fundamental Measurement in the Human Sciences (2nd edition ed.). Mahwah, New Jersey: Lawrence Erlbaum Associates.
  3. Bramley, T. 2007 : Paired comparison methods, in: P. Newton, J. Baird, H. Goldstein, H. Patrick, & P. Tymms (Eds.) Techniques for monitoring the comparability of examination standards. London: QCA.
  4. Linacre, J.M. (1995). Bruce Choppin: visionary. Retrieved 28 January 2013.
  5. Panayides, P., Robinson, C., & Tymms, P. (2009). The assessment revolution that has passed England by: Rasch measurement. British Educational Research Journal. Retrieved 28 January 2013.
  6. Rasch, G. (1960). Probabilistic Models for Some Intelligence and Attainment Tests (reproduced in 1980 with a foreword and afterword by Benjamin D. Wright). Chicago, IL: University of Chicago.

Share this page