Does public confidence in qualifications actually matter?

By Stuart Cadwallader
Published 22 Jan 2015

Coming from someone who works for an awarding organisation, the above question might appear a little petulant. It could read as though I've had enough of the public debate about standards and marking accuracy1 and just want to be left alone to get on with things. I assure you that's not the case I'm not just throwing my toys out of the pram! What I'm really trying to get my head around is the following question:

If we are confident that, as far as possible within the context of the current system, we are providing useful qualifications which are validly and accurately assessed, then does it matter whether or not the public share that belief?

I am fairly certain that the answer is 'yes'. I have two main reasons for believing this:

A continuous effort to maintain public confidence drives improvement. Public confidence is crucial if a qualification is to have value.

Let's take a closer look at each of these.

Driving improvement

Awarding organisations, regulated by Ofqual, attempt to:

maintain grading standards year on year by using the comparable outcomes approach, which also keeps subject standards aligned across awarding organisations make marking and grading as reliable as possible by developing appropriate assessments and mark schemes, and by employing professional examiners who receive high-quality training and monitoring ensure that any concerns candidates or schools have about marking are suitably addressed; a process which, where appropriate, includes re-marking papers.

Would we continue to strive just as hard in these areas were we not interested in public opinion? I would like to think that we would but it seems likely that public scrutiny provides awarding organisations with an additional incentive to meet, and improve on, standards of best practice.

It is worth noting that the role of public confidence in driving assessment practice is not entirely unproblematic. One can imagine a situation where the factors which influence public confidence are out of sync with those that underpin, say, the validity of an assessment.

For example, the use of a ‘level of response’ mark scheme2 may be the most valid approach to marking essay questions which are designed to assess creative or synoptic skills. Such mark schemes require expert examiners to assign an overall mark to an essay based on their best judgement of how it performs against the assessment objectives. The flexibility inherent in this approach is important when evaluating a response holistically. However, even with robust training and monitoring for examiners, it can introduce a higher degree of subjectivity that may make it somewhat less likely that two examiners marking the same paper would award the same mark. Such inconsistency, although infrequent, may be detrimental to public confidence.

However, the alternative would be to use a highly prescribed mark scheme which awarded only specific pre-determined responses (a ‘penny point’ approach). Such a mark scheme may be more consistent, and therefore more appealing to the public, but may inadvertently cause the question to fail to assess the key skills for the subject. The assessment would lack validity even though it may appear to be operating reliably… essentially, it would be reliably assessing the wrong things!

I would argue that such cases are rare and that, on the whole, good assessment practice is positively related to higher levels of public confidence in qualifications. But what would happen if, regardless of how valid and reliable the assessment was, the public lost faith in a qualification?

Grades as currency

Qualifications and grades are clearly very important. They can dictate whether an individual attends a particular university or can compete for a particular job. With that in mind, one could argue that education is a commodity for which grades are the currency3. An individual can use the currency they have earned (their grades) as a way of attaining desirable outcomes in their life.

However, just as cash is essentially nothing but a paper promise (an IOU), the value of a grade depends on whether or not stakeholders believe that it represents what we claim it does. Universities would stop accepting candidates who achieved an A* in their Maths A-level if they no longer believed that the grade demonstrated a suitable level of mathematical ability, or if they believed that the grade had no relationship with how a candidate would perform once at university. They would turn to alternative sources of information, such as admissions tests, to inform their decisions.

In other words, regardless of whether the Department for Education, the regulator and awarding organisations feel that the assessment system is delivering reliable and valid results, people would simply stop using qualifications and grades if they didn’t trust them. Fostering public confidence through the continuous improvement of a high-quality assessment system is therefore crucial to their value.

So, does public confidence in qualifications actually matter? I think it is of fundamental importance.

Stuart Cadwallader

References Billington, L. (2006). Media coverage of examination results, public perceptions, and the role of the education profession. Manchester: AQA Centre for Education Research and Practice. Pinot de Moira, A. (2013). Features of a levels-based mark scheme and their effect on marking reliability. Manchester: AQA Centre for Education Research and Practice. Jones, B. (2011). Regulation and the qualifications market. Manchester: AQA Centre for Education Research and Practice.

Keywords

• Reliability