4 Dec 2013, 12:15am
rant:
by

8 comments

  • December 2013
    M T W T F S S
    « Nov   Jan »
     1
    2345678
    9101112131415
    16171819202122
    23242526272829
    3031  
  • Archives

  • UK OECD PISA scores – the hand-wringing starts again…

    Oh boy, the OECD keeps the bad news coming for Britain’s teens, well according to our press, anyway. I debunked the last attack from the retired colonels at the Telegraph about the OECD scores, but this one seems more serious. I figured I really ought to take a look at the enemy first, and courtesy of the Torygraph I had a go at what PISA call the maths. I am pleased to say that this retired old git managed to do okay in maths. I do have one grouse on the terminology, because when I went to school this sort of thing was called ‘rithmetic – maths started with algebra and trigonometry, your multiplication and division, ie your sums, did not enter the lofty realms of maths, which started at first grade in grammar school, whatever benighted key stage whatever that is called now. It appears that when the OECD and PISA test if you can do maths, they are testing your sums… Meh.

    an ermine did okay, despite my skooldaze being three and a half decades ago

    an ermine did okay, despite my skooldaze being three and a half decades ago

    Even the Grauniad, the paper of choice of the chattering classes and teachers all round, stuck the boot in to the kidz,

    UK students stuck in educational doldrums, OECD study finds

    Cripes. How did our bilious retired colonels take it then over at the torygraph? Straight in the kisser, it seems, and they know exactly what is wrong.

    OECD league tables: UK pupils ‘fail to work hard enough

    There. That’ll learn ‘em, lazy good for nothings. I’m not quite sure I dare see what the Daily Fail has to say on the matter. I had to pinch this next graphic from the Grauniad because the PISA site has barfed, or more specifically the compareyourcontry bit, presumably sagging under the load of parents and journalists tearing their hair out and wringing their hands, respectively.

    1312_OECDLITERACYCHART0212pngNow the first thing is all the papers have chosen the ranking to grizzle about, but if you look at the actual scores they are compressed towards the top. the UK is worst at ‘rithmetic, but 60 points are between us and the top score versus 85 to the bottom score, and yet we’re a bit more than halfway down there. For all that, it’s fair dos to South Korea, Japan, Finland and Poland, who show and excellent score and excellent balance. The Swiss are all round better than the UK scores but are less balanced, and  other countries are less balanced than the UK result, which surprises me.

    I don’t actually find this so terrible either, Perhaps I am complacent, but I think the UK comes off okay in terms of the balance of the education achieved by school leavers. It’s an easy headline to yell that the UK slips below 20th in ranking (I’m not quite sure how the Graun got the UK at 26, 23 and 20 in maths, reading and science as it doesn’t really tally with the numbers they’ve lined up on the right, maybe they are showing the rotten UK education of their interns while the staff went down the pub).

    I can’t face rekeying in the data to place this on a linear scale to show the compression towards the top. I’m also surprised at some of the results – take Israel for instance, which has a seriously good high-tech industry, in particular some areas of software, codecs and compression, the poor showing in science and maths is odd. The US shows a poorer result than the UK, so God knows how they get to be the largest single economy in the world with such an absence of smarts, and we presume that the Swedes were generally out to lunch when the PISA team came round ;)

    Obviously the UK could and should strive to do better. It probably is fair to say that some Asian countries place a very high premium on studying even at school level – in which case trying to replicate those systems will rub up badly against more individualistic Western cultural preferences. More importantly we need to ask what we want of our education system – in general it seems to be creating citizens that can add value in the economy. Hopefully PISA score line up with that.

    Getting better scores really shouldn’t be that hard. I could manage to do my bit for Britain score, despite there being an presumption that skills start to decay immediately on leaving university (from the OECD paper referenced in the earlier article). I’d be a little bit disturbed if the 50+ year old ermine’s education had decayed back to the level of a 15 year old simply due to the passage of time! US Slate’s article hints that an excessive focus on testing and test results hampers PISA scores, as they are also tests of inference and analysis, whereas from what I’ve seen of modern school tests (it isn’t a lot) is that the questions strike me as spoon-fed, no inference necessary. And yet inference is necessary to apply knowledge to the real world and turn it into wisdom.

    So once again, leave them kids alone ;) it could be better, but it isn’t dire. In many ways educating our schoolchildren in some key intangibles would be a better win – doing better at deferring gratification, and the fact that often in life you have to stick at something to actually get anywhere could make them a lot more effective. I am not so sure that the problems of Britain’s young are all placed at their education, it is their upbringing and the values their parents seem to fail to instil in them that seem to be thwarting effective behaviour when they grow up. Perhaps the excess focus on testing and metrics is trammeling thought too narrowly, it wouldn’t be the first case in recent times where an excessive focus on process and metrics delivered lots of what we say we want but failed to deliver things we couldn’t easily measure but really do want.

    Although I am with Lord Kelvin’s grouse about qualitative information for things like battery life, I suspect in business and in education we have too much measurement of the things that are easy to measure and are taking our eye off the ball as far as the good things that are hard to measure. Metrics without values is a nasty road to hell IMO despite the good intentions.

    After writing this I came across this which has an interesting angle on the remarkable success of the Asian countries at improving things greatly relative to Western countries. Although he considers that the UK is very average, he says a key difference with Asian countries is

    The OECD argue that the single biggest reason why the Far East does so well is that they do not have the fixation with innate ability that many Western countries have

    Now I do have this fixation, though it was partly a result of my selective experience. Selective education worked for me, because it got rid of the chavs kicking holes in the classroom walls. In educational theory if you throw enough adults at the problem you can discover what is troubling the chavs enough to raise their self-esteem so they don’t kick the damn walls in and crap on everyone else who are cowering trying to avoid getting a thump from these little shits.  1960s Britain was not rich enough to do that, and selective education at least save some people, the young Ermine included, though it let the less able go hang 1

    So pardon me if I am thoroughly of the opinion that you have to triage kids in education because otherwise some of the little blighters will wreck the life chances of others in order to express their precious little selves. Too much of the anti-selective narrative is about the life chances of the ones that failed the 11+ being wrecked. I’d find it more convincing if it acknowledged that lumping everybody together has its issues too – it only takes a few little bleeders to stiff the chances of everyone in a class of 31. Despite that delightful experience I do accept that the plural of anecdote isn’t data, and theoretically perhaps nonselective education might work. I don’t believe it, but because I have nothing to do with education what I think doesn’t matter so it doesn’t harm any kids’ education ;)

    I do agree that pretty much anyone could be taught how to achieve a decent score in PISA. As I said above, the maths section in PISA is not maths, it is arithmetic, and unless you are the wrong side of Boris’s 16% – and maybe even into that, you can be taught how to do basic sums, and perhaps how to apply this in the real world.

    However, getting decent PISA scores isn’t going to make you advance human knowledge. At the higher levels, innate ability, combined with competent teaching, is where the leading edge will out. My maths 2 is relatively poor despite doing fine with PISA. I flunked university second year maths because I had no talent for it – and solving differential equations was where I ran out of road. There are some aspects of maths that you either get, or you don’t. Yes, reading the old textbooks now and with no pressure I might be able to comprehend it a little better, but I will never be good at it. I was one of the early users of the DOS version of Mathcad at work because I knew I was weak there, so I looked for ways to work round it 3. And when it comes to knowledge, I’d say the few percent at the leading edge are what matters to push a technological economy forward. I’m not sure that we should give up this fixation with innate ability at the highest levels, although I do take the point when it comes to school.

    It probably costs an awful lot of money to drag the bottom end up to PISA standards, but it appears it can be done. Whether that is a worthwhile economic proposition for society I don’t know, but if it’s considered so in Asia and they get the results then it’s worth considering. However, I figure they have far better discipline in schools than we have – the crowd control aspect of teaching due to piss poor parental values in many cases could stymie attempts to bring everyone up to a decent level of PISA attainment. And before my money is spent in nonselective attempts to bring everyone up I’d like something to be done about discipline and attendance. It may be that to achieve Asian level attainment we will need to spend a hell of a lot more money and violate some of the rights of some parents to not give a shit. Which may not square with Western individualism and the right to self-determination and pursuit of happiness.

    Notes:

    1. This wasn’t as harsh in those days as it would be now, at the time the economy had jobs for the non-academic in abundance.
    2. maths is knowing what div and curl are, how to wrangle tensors, calculus and statistical methods. It isn’t working out the average speed of a kid’s pushbike :)
    3. that doesn’t always help, but it was good enough for analogue filter design, nowadays you’d use computer programs, simulation or more likely do it in DSP
    4 Dec 2013, 9:51am
    by BeatTheSeasons


    I followed the link from the BBC new story and did the test. Question 6 was:

    “Helen rode her bike from home to the river, which is 4 km away. It took her 9 minutes. She rode home using a shorter route of 3 km. This only took her 6 minutes.

    What was Helen’s average speed, in km/h, for the trip to the river and back?”

    You only need to know that speed is distance divided by time to answer this question. So the answer is 7 kilometres divided by a quarter of an hour which is 28km/h.

    According to the OECD website only 3% of British 15 year olds achieved level 6 compared to 31% from Shanghai.

    I’ve got to say that after over 10 years of compulsory schooling I find it shocking that only 3% can answer this trivially simple question.

    Have I misunderstood something about the test?

    @BTS You and I did okay with that because inference and deduction were a part of how our exams were designed. Okay, I presume we also turned over the old grey matter since then occasionally so may have learned more along the way, but it’s not necessary for PISA. Those exams were designed to be marked by humans who would track the reasoning so even if you got the answer wrong if you worked it out in the right way up to a point you got some marks. That just doesn’t happen now, because it was considered unfair to test inference and deductions at the same time as the ‘maths’ process. Sort of along the lines would you smash your calculator just because it couldn’t do that question without your help to turn it into 1/(1/4)*7?

    I believe from the little I have seen of UK school tests, they are trying to split out the process of deriving an average – they would be couched in terms of ‘Helen travels 7km in 15 minutes, what is her average speed?”

    In doing all the testing, they have eliminated what used to be called inference and induction from schooling. I pinched that theory from the US Slate article I referred, which also asks what the bloody hell is going wrong with us in the US, where they seem to have a similar focus on metrics and testing? My teachers had far greater freedom to go off-piste in the classroom to follow trains of thought, well, once the chavs had been eliminated ;) Someone I know recently took a look at teacher training in a local well-thought of school, Kesgrave High, and in one afternoon of seeing the amount of time devoted to crowd control knew this was not for her – this has changed dramatically from my schooldays. Although we caused trouble often enough it was sporadic, and deep down we knew that we needed to learn at least some of this stuff!

    I would have easily been able to do that PISA example at 15, indeed it disturbs me slightly that only 31% of those from Shanghai can do it, because these are the people who will be engineering our world and fighting the technical challenges of the first half of this century. However, this doesn’t detract from the conclusion that the UK is reasonably average and also reasonably balanced compared to its peers.

    Do we need to get everyone up to Asian standards? I’d glad to say that after a few beers in their gentleman’s clubs, the retired colonels at the torygraph are easing back of the rhetoric and wondering if there may not be unintended consequences after all I know AEP is a rabble rouser, it always feels uncomfortable when I think he may have a point!

    […] One of the biggest challenges for Government is the improvement of education. Here is an example of why we have such an important role to play, from a comment on this site […]

    Sadly my home country, Wales did even worse than everyone else.
    Probably because if there had been an exam in alcohol consumption we’d have come top.

    7 Dec 2013, 9:52am
    by ermine


    @Sara – early starters, too – PISA is targeted at 15-16 year olds I believe ;)

    7 Dec 2013, 8:21pm
    by AbeBird


    According to the Israeli press children who were tested in hebrew had a math results of 489 while children who were tested in Arabic scored 388 . That means that the gap between Arabs and Jews is 1 SD (100 point).

    Consider that Ashkenazim/Mizrahim ratio among Hebrew testers is 50/50 , and that Ashkenazim typically score 0.75 SD (75 point) then Mizrahim (e.g on the Israeli SAT) , then Ashkenazim score in math is about : 489 + 75 * 0.5 ~ 521 .

    Anyway, PISA isn’t precise but isn’t useless either. No methodology designed to compare countries with completely different cultures and education systems will ever be perfect. The OECD acknowledge that their rankings aren’t exact and it makes more sense to look at buckets of countries who scores similar / close cultures.

    8 Dec 2013, 3:44pm
    by ermine


    @AbeBird – interesting, though I’m not necessarily sure that the difference between children tested in different languages necessarily can be extrapolated that far ;) However, I’d be the first to acknowledge I don’t have any real understanding of Israel’s education system. The clear leadership in some scientific/technical circles could be a result of prioritising university education or indeed favouring academic excellence.

    Foreigners such as me will naturally get sample bias because we’ll see the results of the best, because these are the people out there producing goods and services that we want to buy!

    9 Dec 2013, 5:40pm
    by Caroline


    I’m intrigued by results such as the one for Slovenia, which suggests their kids are poor at reading but soemhow still manage to do well at science and maths! Given the long-winded nature of maths question given above, how did they manage that?

     

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>