Maui Street has a piece by Kelvin Davis summarising John Hattie’s research on 138 possible measures for improving education, which is the work that National and Treasury – supposedly – based their education reforms on:charter schools, league tables, performance pay, larger class sizes, improving teacher performance.
Charter Schools have an effect size of 0.20, or the 107th out of the 133 strategies that have some positive effect. Charter Schools are therefore an extremely pointless and expensive strategy.
There are still 40 strategies that are deemed pointless, but, are still more effective than Charter Schools.
Quality of teaching is 56/138. Professional development for teachers was 19/138. Teacher League tables and performance pay simply don’t register in Hattie’s research.
(I should also point out that very few of the studies in Hattie’s meta-analysis were conducted in New Zealand. And we have a very unusual education system: it performs better than most OECD countries, even though we spend less money on it. So things that might be ‘low hanging fruit’ that yield great gains in other education systems might have no effect here, because we’re already performing at a very high level.)
I was also interested to see that Hattie’s measurement of class size – the much vaunted claim that ‘it makes no difference’ (106/138) was in regards to classes that reduced in size from 25 to 15. National planned to change the range from 23-29 students/teacher to a set ratio of 27.5 students/teacher. They still think it was the correct thing to do, even though the very research that they cite fails to endorse this view.
But maybe they’re right! Who knows? Well, we have this amazing way of figuring out if things are ‘right’ or ‘wrong’ – it’s called the scientific method. You come up with a theory, test it and see what happens. If I was the Treasury Secretary, or the Minister of Education and I wanted to find out if these things did make a difference I’d run right out and set up some small scale trials in some classrooms, and measure the results, comparing them to normal classrooms in the same school. Seems to work? Let’s implement it across a bunch of different schools across the country and monitor them, and so on.
That’s how every other scientific discipline works, but for some reason (most) economists don’t roll like that – especially not the ones we have running our Treasury. When it comes to economics and public policy, the procedure is that economists come up with a theory, assume that they’re correct, even though they’ve made some really, really stupid mistakes in the very recent past and try and impose their theory on vital social infrastructure like the national education system without any empirical evidence whatsoever.
Unless we see a lot more humility, and a lot more actual science – in terms of empiricism and falsification – we should simply stop listening to those people.