To ask Her Majesty’s Government what assessment they made of external expert advice prior to the use of the algorithm to determine A-level results for the 2019/20 academic year.
My Lords, the independent qualifications regulator, Ofqual, is responsible for securing qualification standards and promoting public confidence in regulated qualifications. As part of the development of the grading system introduced in place of exams this summer, Ofqual drew on the advice of experts from the exam boards and convened an external group of well-respected assessment experts to advise on the principles, main features and details of various aspects of the standardisation model.
My Lords, expert advice early on identified the algorithm as flawed and particularly damaging for state school and disadvantaged pupils. We have been told that the Secretary of State was fully in charge of his department throughout this debacle, yet two senior officials have resigned, which is outrageous. When I tabled this Question four weeks ago, I thought that he might have fallen on his sword by now—but no. So can the Minister confirm that the Conservative Government, with the exception of the noble and learned Lord, Lord Keen, have abandoned the principles of ministerial honour and responsibility?
My Lords, on the standardisation model, Ofqual is a non-departmental body. It is important in principle that our examinations are not subject to government interference. While the department was in contact with Ofqual during this process, the decisions made on the algorithm were Ofqual’s. That respects the appropriate relationship between a department and independent bodies such as Ofqual.
Algorithms were a failure this year; they will not happen next year. Today, there are thousands of students—possibly even hundreds of thousands of students—who are not at school because of the lack of testing. On Monday, Liverpool University Technical College had to send home all year 10, year 11 and year 12 children—hundreds of children, who may be out for 10 days. This will happen all over the school estate. There will not be a level playing field of attendance records for students, and it will not be their fault. It is therefore very unfair to test them by written exams next year, because each student will have a different level of attendance. The Government should recognise that teacher assessment will be needed this year, in which case they should issue guidance to teachers now on the state of reports they will have to keep on each student, not only on attendance but on progress. If written exams happen next year, the brightest children will do well and the disadvantaged will do very badly. That is simply not fair.
My Lords, every Tuesday, the department publishes attendance data. As of yesterday, nearly 88% of students in state-funded schools and institutions were in school. The guidance published before the summer holidays made it clear to schools that by the end of this month they must be able to stand up remote education in the eventuality that pupils are sent home in these circumstances. We are working with Ofqual, which is looking at the arrangements for next year’s examinations.
My Lords, the algorithm for predicting A-level results this summer was clearly too harsh, leading to overcompensation using teacher predictions and subsequent difficulty finding college and university places in subjects such as medicine. If the physical sitting of exams again proves impossible this summer, will the Minister ensure that the timetable for publishing results allows more time for the better matching of teacher and improved algorithm predictions with the availability of places in higher education?
My Lords, on the important issue of the placing of students—particularly for A-levels, which are more often progression exams—the noble Lord will be aware that the Government, working closely with higher education institutions, lifted the cap on certain courses to raise capacity. The most recent figures are that 89% of students who received a grade increase have got their original offer, their insurance offer or an offer at an institution with the same tariff as their original offer.
My Lords, the most frustrating element of the algorithm was that it assumed that schools could not improve on previous years’ best performance. That seems contrary to what any Education Minister should believe about the power of schools to improve and change children’s lives. Did that element of the algorithm come from an external expert? If so, why was it accepted?
My Lords, as the noble Baroness will be aware, Ofqual consulted on the methodology and what aspects to include in the algorithm. The issue of what we termed “outliers”—highly performing students in institutions which have previously not performed well—was raised and was in the balance; students who might be affected in that way could be put right through the appeals processes. However, when the balance became such that the level of anomalies outweighed this, the more just situation became to use teacher assessment grades rather than the algorithm to assess grades.
My Lords, could the Minister give us an absolute assurance that this algorithm or anything like it will never be used again? That is something we should hear today.
My Lords, all four nations of the United Kingdom attempted to use this method. At the moment, the Office for Statistics Regulation, which is part of the UK Statistics Authority, is looking at the algorithms used for all four nations. However, it is intended that exams will go ahead this summer.
My Lords, the Secretary of State, the Department for Education and Ofqual were all warned by Cambridge Assessment of serious flaws in the grading of exams two weeks before A-level results were published, yet no action was taken. Much more seriously, the Royal Statistical Society has said that the issues with the algorithm could have been avoided had independent expert advice been taken. As far back as April, the society highlighted to Ofqual the problems coming down the road and suggested the establishment of an advisory panel involving independent statisticians to deal with them. Can the Minister explain to the hundreds of thousands of young people whose lives and education have been disrupted unnecessarily why that course of action was not taken?
My Lords, a member of the RSS was present on the expert advisory group at Ofqual, which I have already outlined. Ofqual tested 12 different models of the algorithm. During the algorithm’s development, there were various meetings between the department and Ofqual, and we were assured that any irregularities in its application could be put right through an appeals process. We responded when an issue arose in Scotland around its use of an algorithm.
My Lords, will the Government conduct research into the extraordinarily large difference between predicted grades and actual grades at A-level, so that we can understand why deprived children fall below predicted grades so often and do something about it?
My Lords, there was a rise of about 12% in the top grades awarded this summer. We are not in a position to go behind the teacher assessment grades. The only appeal available to students is on the basis of administrative error in giving those teacher-assessed grades to the exam boards.
I draw noble Lords’ attention to my relevant interests in the register as an adviser to a decision science company. Does the Minister agree that the approach taken to determine the outcome of this year’s A-level results clearly demonstrates that using an algorithm or human expert judgment in isolation is flawed, and that highly complicated decision-making by government needs to embrace decision science, which seeks to exploit the right balance of artificial intelligence and human judgment?