The Dim-Post

September 23, 2012

Well below standard in analysis

Filed under: education — danylmc @ 9:08 am

The HoS trumpets:

Primary schools have disclosed controversial data about pupil achievement, with the surprise revelation that children in bigger classes and bigger schools get better grades.

The Herald on Sunday has conducted a comprehensive survey of schools’ national standards results, before the Ministry of Education publishes them this week.

At schools with fewer pupils for each teacher, around 70 per cent of children are achieving national standards in reading, writing and arithmetic. But at schools with more pupils for each teacher – in effect, bigger classes – the pass rates rise to about 80 per cent.

So too with school rolls: the highest proportions of children achieving or exceeding national standards are at big schools.

I’ve just checked the data – and they’re absolutely correct! There’s a strong linear relationship between class size and National Standards results. Here’s a scatter plot for reading results vs class size.

Bigger classes are better! Just like Treasury and the government said! But wait, what’s that cluster of very small classes with very poor results down there in the lower left quadrant? Well, those, which have well below standard in every single category are:

Blomfield Special School & Resource Ctre
Waitaha Learning Centre
Wilson School
Kea Street Specialist School
Parkside School
Hamilton North School
Sir Keith Park School
Every single one of which is a school for children with disabilities. Take them out of the dataset and see what happens to the trendline:

Most of the other low-quality, low ratio schools I note that they’re predominantly low-decile schools in rural areas. So now we know that very poor, very small schools aren’t that great for the kids.

Update: updated the charts after the statisticians chastened me for improper plotting, and added the regression co-efficient by (actually rather popular) request.

About these ads

20 Comments »

  1. Great stuff. What’s the R^2 and slope?

    Comment by Mike Dickison (@adzebill) — September 23, 2012 @ 9:35 am

  2. there’s a cluster of 3 large class schools with middling results that don’t appear in the second graph – if they were in there, the trend would be even weaker – I think. What’s the scale on the horizontal?

    Comment by Deano — September 23, 2012 @ 9:45 am

  3. The x scale works like this:

    add

    Well below x -1.5
    below x -1
    standard x 1
    above standard x1.5

    together for each category. happy to hear from statisticians on how this might be a terrible way to quantify ns results

    Comment by danylmc — September 23, 2012 @ 9:50 am

  4. The r^2 value and slope would be important to know, but also the p-value for the hypothesis that class size is a linear-predictor. I’m wondering if non-linear (Poisson) regression would be more appropriate here though. The number of students per teacher are counts.

    Either way, the signs aren’t good for establishing a link!

    Comment by Shane Field — September 23, 2012 @ 10:39 am

  5. Thank you for that analysis.
    The danger has started- crap analysis of crap data. by crap analyzers.
    How can you get this published in a wider forum to counter the BS?

    Comment by Dv — September 23, 2012 @ 10:52 am

  6. I would also like to see the decile ratings pulled out of that data. I wonder if lower decile schools may have lower staff to student ratios due to increased teaching/support staff – I don’t know I am just thinking that this might be affecting the numbers – someone who knows more about how the system works would be able to say?

    Comment by Shane Gallagher — September 23, 2012 @ 11:12 am

  7. a teacher tells me lower decile schools get more funding per pupil. they can choose what to spend it on. some might choose reading recovery, for example….

    Comment by lucyjh — September 23, 2012 @ 11:18 am

  8. So now we know that very poor, very small schools aren’t that great for the kids.

    My god. Praises be to Stuff for selflessly publishing this mind-blowing data, and the Herald on Sunday for … I don’t know.

    Comment by QoT — September 23, 2012 @ 11:27 am

  9. “Most of the other low-quality, low ratio schools I note that they’re predominantly low-decile schools in rural areas.”

    It’s so easy to attract the high quality teachers to poor rural schools. Also special needs kids in rural areas have, lets see, a choice of one school often and said schools can really cope with them. *sarcasm alert*

    Comment by Michael J. Parry — September 23, 2012 @ 11:49 am

  10. Ignore all of that.
    Look at the “BIG” class sizes!?
    The large cluster of “performing” schools are in that 20-25 students (which as a teacher is what I’d call ideal), not the proposed 27.5 which would increase to 30-35+ the higher up the school you go! So none of this looks like it suggests what we’d consider as a “big” class size.

    Comment by alingham — September 23, 2012 @ 12:25 pm

  11. Given that students aren’t forced to go to their local schools, all we’re really seeing is that people with the resources to place their children where they wish tend to shift high-scoring kids to big wealthy schools. Well, really we’re not seeing either, because there’s all sorts of other variables unaccounted for.

    It could have made interesting longitudinal data if the school names had been removed, only people will shift kids around in response to all this nonsense, so unless they track individual children, rather than “schools” of them, it won’t be be much use now.

    Comment by tussock — September 23, 2012 @ 3:17 pm

  12. It looks to me as if there isn’t a significant relationship. So … class size is not a performance indicator – either way! So class size is NOT the issue or indicator some people want it to be. Now, how about teacher quality?

    Comment by David from Chch — September 23, 2012 @ 3:59 pm

  13. “Now, how about teacher quality?”

    Well first we’ll need to come up with some way to turn the complex process of teaching into some simple numbers so we can set up some league tables and compare them.

    Yay Science!

    Comment by nommopilot — September 23, 2012 @ 5:00 pm

  14. Well if you *must* do a simple linear regression you should switch your x- and y-axes, which gives you a different line. It may or may not make sense to predict achievement from class size, but it makes no sense to predict class size from achievement.

    But it’s probably neither simple nor linear.

    Comment by bradluen — September 23, 2012 @ 5:37 pm

  15. I’m not statistician ( I can’t even spell the word…), but given the graph for performance for decile rating, if you were to standardise every school to a decile 5 school, how would the graph go then?

    Comment by Michael — September 23, 2012 @ 11:04 pm

  16. You need to do some binning somehow to turn it into a histogram – as a non-statistician engineer, I look at the envelope of the updated chart and think emperically:

    “If I want a school to have a decent reading score, such as >0.5 on the graph, class size should not exceed 25. Optimum class sizes appear to be between 16 and 22 – this has the greatest density, i.e. most of the good results lie between those values”

    But perhaps, as a non-Herald reader, I would come to different conclusions than them.

    Cheers,
    FM

    Comment by Fooman — September 24, 2012 @ 8:00 am

  17. You need to do some binning somehow to turn it into a histogram

    Either that or a multiple regression – I’ll see if I have some spare time later this week.

    If I had the luxury I’d create a hashmap of the schools, with the value as a ‘School object’, and make it publicly available. If anyone else is up for this let me know and I’ll link to so others can use it.

    Comment by danylmc — September 24, 2012 @ 8:14 am

  18. Just eyeballing the data it is obvious there is no statistically significant correlation between class size and achievement in reading. Can you give us the result of the t-test on the explanatory variable. I bet the null hypothesis holds.

    Comment by Doug — September 24, 2012 @ 11:03 am

  19. Arrrrgh! Why are you even trying to analyse this data. The data is so unreliable it’s essentially worthless. There is no moderation and no consistent assessment just a bunch of aspirational statements.

    Comment by Phil — September 24, 2012 @ 3:07 pm

  20. Yes a t-test please

    Comment by Shane Field — September 24, 2012 @ 9:50 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Rubric Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 403 other followers

%d bloggers like this: