Leaning into a PISA world

Sandra Johnson of Assessment Europe considers the impact of the OECD’s Programme for International Student Assessment on national qualifications.

International comparisonsIn mid-October 2012 the French President François Hollande presented to his country the eagerly-awaited report on a far-reaching and historic national consultation on education. The fundamental and broad-ranging question posed to stakeholders was essentially ‘What should schooling be all about?’  Unsurprisingly perhaps, many of the resulting recommendations focus on overturning the policies of the previous Sarkozy government. Others, though, aim at new system reform, perhaps the most welcome being the abolition of the long-standing and unpopular practice of redoublement, in which low-achieving students are held back to repeat years with no clear personal benefit and no system benefit either.

I offer this news from my host country simply because it is one of the most recent examples of what has become widely known as ‘PISA policy impact’. The French pronouncement followed hot on the heels of Michael Gove’s announcement about finally abandoning the GCSE, an initiative which if not initially inspired by PISA results has also certainly been heavily influenced by them. So what exactly is PISA?

All about indicators

PISA, the OECD’s sample-based Programme for International Student Assessment, was launched in 2000. The intention was to fill an indicator gap by providing OECD countries with information about system output (principally student achievement) to put alongside information about input (such as human and financial resources) and process (system organisation).  PISA focuses on 15-year-olds, with achievement assessed on a three-year cycle in three key ‘life skills’ domains: ‘reading literacy’, ‘mathematical literacy’ and ‘scientific literacy’. Questionnaire information is routinely gathered and increasingly used to explore relationships between student achievement and student, school and system characteristics.

The new availability of comparative information about system outcomes at the end of obligatory schooling, provided by such an authoritative body, could have been predicted to have an important policy impact internationally. But the OECD itself might have been surprised by the immediacy of the reaction, and by the extent and importance of PISA-inspired, or at least PISA-supported, educational reform over the past decade.

Many countries, in Europe and elsewhere, experienced what has become known as ‘PISA shock’ when the first PISA results were published, despite the fact that in many cases concern about national performance was unwarranted. From the earliest survey reports, country league tables based on a technically stretched score scale have tended to exaggerate inter-country differences that in reality could be small or non-existent.  In response, governments everywhere have embarked on reviews of every aspect of their provision, with policy initiatives following in curriculum, instruction and assessment. 

The range of reaction

At least partially in response to PISA, several countries have been reviewing and reforming their existing national curricula for the duration of obligatory schooling – England is just one example. Others are in the process of introducing common regional or national curricula where none previously existed: notable examples include Germany and Switzerland, the former striving to align the new curriculum as tightly as possible with the ‘PISA curriculum’ (ironically, since PISA by design focuses on process rather than content in an attempt to transcend national subject curricula). National assessment programmes have been introduced where none existed previously – countries as far apart as the Czech Republic and Australia join France (currently remodelling its fledgling system), GermanyNorway Sweden, Switzerland and many others in this respect – and where there is formal assessment in the lower secondary school this is being scrutinised, as in England.

School leaving qualifications are not untouched. The previously trusted teacher assessment that has long underpinned school leaving qualifications in some countries – for example, Norway and Sweden – has been shown wanting, and national tests are being introduced or given greater weight to address comparability concerns. With a similar motivation, in 2011 the Czech Republic’s examination-based qualification, the maturita, saw the addition of a standardised national examination component to complement the school-based assessments. For the most part, centrally-controlled examination-based school-leaving qualifications, including the French Baccalaureate, have so far escaped PISA impact, but they are not in principle immune.    

The globalising influence of PISA is strong and irresistible. So much so that it is now inconceivable for any debate about educational system effectiveness not to include reference to PISA survey results. Many of the policy initiatives that have followed have been welcomed by stakeholders in the countries concerned, but others less so.

The question the international assessment community must now ask is how can we ensure that politicians and policy makers interpret PISA results intelligently, and use them responsibly to suggest appropriate policy directions for system improvement, whilst resisting the inevitable temptation to cherry pick findings to promote pre-existing political agendas?

Sandra Johnson is based in France as co-Director of Assessment Europe, an independent assessment research and consultancy company, and Visiting Fellow in the University of Bristol’s Graduate School of Education.

Share this page