Sunday, 26 July 2009

Letter to UK Statistics Authority


26th July 2009

Richard Alldritt

Head of Assessment

UK Statistics Authority

Statistics House

Tredegar Park

Newport

South Wales

NP10 8XG

Dear Mr Alldritt,

Conduct of the Review of Elective Home Education

On the 11th June 2009 the Department for Children, Schools and Families published its Review of Elective Home Education in England, which was conducted by Graham Badman. In at least two instances associated media coverage alleged that twice as many home education children were known to social care compared to the rest of the population, a claim that was attributed to the report’s author, although it is not made in the published report:

‘The reforms are necessary because twice as many home educated children are known to social services as the normal school-aged population under current arrangements, the report revealed.’

(Woolcock, N. (2009) ‘Home education parents to face council inspections, Times Online, 11th June, Retrieved from http://www.timesonline.co.uk/tol/life_and_style/education/article6480288.ece on 26th July 2009.)

‘Children educated at home are twice as likely to be on social services registers for being at risk of abuse as the rest of the population, the head of a government inquiry into home education said yesterday. … Graham Badman, the former director of Kent County Council's children's services, headed the review. He said the ratio of home-educated children who were known to social services was "approximately double" that of the population at large.

(Anon. (2009) Children educated at home more at risk of abuse’, The Independent on Sunday, 12th June, Retrieved from http://www.independent.co.uk/news/education/education-news/children-educated-at-home-more-at-risk-of-abuse-1703220.html on 26th July 2009.)

That the ‘twice as many’ is an official statistic is confirmed by a Department for Children, Schools and Families Freedom of Information response that provides an insight into how the estimate was calculated. The Freedom of Information request was made by Mr. S. Mckie and the response made by Ms S. Thomson on the 24th July (a copy is at http://www.whatdotheyknow.com/request/local_authority_evidence). The response is headed Annex, although it is currently unclear to which document it was originally appended. Our concern is that both the press statement and the ‘Annex’ breach the Code of Practice for Official Statistics, in particular Principles 4 (sound methods and assured quality) and 8 (frankness and accessibility), and the Civil Service Code for the following reasons:

1. Quoting to the media the ratio of the two proportions, without stating the actual estimated figures, arguably conveys the impression that the difference is more severe than the actual percentage estimates suggest: 6.75 per cent for home educated children and 3 per cent for the population. A more honest and impartial statement would have involved giving the actual figures and/or mentioning that the difference is only nearly 4 percentage points.

2. The interpretation and presentation of the variables used is misleading. As the above quote from The Independent shows at least one organisation completely misunderstand the nature of the data – ‘known to social care’ is a different administrative category from being registered ‘at risk’. The Department for Children, Schools and Families does not appear to have insisted that The Independent issue a correction to the misleading impression that their attribution to Badman will have created amongst the public.

The Freedom of Information response refers to children known to social care including Section 17 and 47 enquiries under the Children Act 1989. Given the focus of the Review on child safeguarding the inclusion of Section 17 cases in the calculation of the estimate is inappropriate. Section 17 covers the provision of services to children in need and is not an indicator of the risk of child abuse, it includes, for instance, parents requesting a service such as respite care for a disabled child.

Section 47 cases include child protection inquiries but also referrals to social care irrespective of whether or not child abuse is subsequently established. It is likely that home educators are, wrongly, over-represented amongst Section 47 referrals because (concerned) third parties, unaware of the legal right to educate at home, mistakenly contact social services. In some cases the parents will be ‘known to social care’ but there is no suggestion of their children being at risk of abuse. The Section 47 figures, therefore, over-estimate the number of at risk cases amongst the home educating community.

These difficulties with interpreting ‘known to social care’ statistics that arise from Sections 17 and 47 are not mentioned in the Annex, and it is unknown if Badman tried to convey them to the press.

3. A related point that the data may not be comparable. The 3 per cent population estimate relates to individuals aged 5 to 16 years.[1] It is unclear whether the sampled ‘known to social care’ data counted individual children or enquires. It is known that the Section 47 data refers to enquires not children. Each enquiry in any one year for a particular child is separately recorded.[2] Moreover, it is unknown whether the ‘known to social care’ was similarly restricted to children aged 5 to 16 years, or whether, for instance, it included younger children.

4. There are a number of issues arising from the calculation of the ‘known to social care’ estimate that should have been highlighted, at least, in the Annex.

a. The denominator for the estimate assumes a median of 139 registered home educated children per local authority; giving 20,850 (139*150) nationally. The Freedom of Information response rounds this up to 21,000 whilst Badman rounds it down to 20,000 in the report of the Review. The median value of 139 comes from the survey of 90 local authorities. How representative these 90 councils are of all local authorities is unknown. No attempt appears to have been made to correct for response bias.

b. The estimated number of registered home educated children known to social care appears to be obtained by applying a percentage (6.75 per cent) to the estimated total number of registered home educated children (see a above). There are three issues here.

First, according to the Freedom of Information response the percentage applied (6.75 per cent) is the median value taken from a small sub-sample of 25 local authorities (17 per cent of all English local authorities). Again, the representativeness these 25 local authorities is unknown; although our estimate of the mean number of ‘know to social care’ for these authorities (19 compared to 9 for the country has a whole) suggests that they are atypical and have an above average number of cases. For such an unrepresentative sample, even the use of the median percentage is likely to be an over-estimate of the proportion that should be used in national estimates. Thus grossing-up using 6.75 per cent is likely to produce an invalid over-estimate of the number of home educated children ‘known to social care’. Yet no qualifications to the estimate are mentioned in the Annex, and whether Badman tried to convey them to the press is unknown.

Secondly, the Freedom of Information response states that the estimated number of home educated children know to social care is 1,350. However, if the 6.75 per cent was used to derive this estimate then the base used was 20,000 (a figure given in the Review) and not the Annexes’ own population estimate of 20,850 (which would give a higher estimate of 1,407). It’s not clear why a base of 20,000 has been used rather than 20,850 (or a rounded up 21,000).

Thirdly, and as the Freedom of Information response acknowledges, the estimated size of the population is an under-estimate. Many home educators are unknown to local councils. The Review says that the estimated number is 20,000 but it is likely to be double that figure, if not more, possibly up to 80,000 children’. If more home educated children were ‘registered’ with local authorities – and clearly such families exist – then the median proportion of 6.75 per cent for known to social care would be lower; as the base for such calculations would be a larger number. In these circumstances the difference between the percentages for home educated and the population known to social care would narrow. The estimated proportion of known to social care for each of the 25 local authorities should have included a range of estimates using higher estimates of the number of home educated children in each area.

Both the Review report and the Annex ought to have included a range of estimates using different nominators and denominators

You should be aware that we have many other concerns about the conduct of the Review, that the Children, Schools and Families Select Committee has instigated a brief enquiry into the conduct and recommendations of the Review, and that this letter and your response are likely to be blogged at http://maire-staffordshire.blogspot.com/.

We are writing because we believe that the Review does not reach the standard required for such a radical change in the law relating to home education, and that its shortcomings are inevitably reflected in its recommendations. We look forward to hearing from you.

Yours sincerely

Prof. Bruce Stafford

Mrs Maire Stafford.

(sent by email)



[1] The source for the estimate appears to be: DCSF (2006) Children in Need in England: Results of a survey of activity and expenditure as reported by Local Authority Social Services’ Children and Families Teams for a survey week in February 2005: Local Authority tables and further national analysis (Internet only),Statistical Volume, National Statistics, Retrieved from http://www.dcsf.gov.uk/rsgateway/DB/VOL/v000647/vweb02-2006.pdf on 26th July2009.

[2] Department for Children, Schools and Families (2009) Guidance Notes for the Completion of Statistical Return CPR3 Referrals & Assessments of Children and Young People who are the subjects of Child Protection Plans (on the Child Protection Register) 1 April 2008 to 31 March 2009, Retrieved from http://www.dcsf.gov.uk/datastats1/guidelines/children/docs/CPR3GuidanceNotes0809.doc. on 26th July 2009, p.9.

11 comments:

Elizabeth said...

You two are wonderful!!
Hoping someone there takes this seriously.

Sally said...

Fantastic. I've been trying to work through this stuff too. You've not mentioned section 37 which is in Badman's footnote 2 in annex 5. Not sure if it was applicable.

Also, section 17 'known to's may include siblings of 'children in need' due to disability, etc. It appears to.

Serious Case Reviews (included in section 47 and relating to enquiries rather than individuals?) can be enquiries following death or harm, or following mishandling of a case, according to the recent critique by Ofsted. May Badman be using stats that include numbers of badly handled cases as if they are harmed children?

I'm still reading through and sending comments as I work.

Sally said...

sorry that should be footnote 1

Sally said...

I have been in considerable contact with the Independent regarding this article. (I'm hoping I've saved their replies!) They claim their journalist has correctly quoted Badman from his press conference. I wonder if we can get a transcript of the press conference?

Sally said...

I've been writing this to someone who is working on a press release. I think we need to be careful to not accuse Badman of deliberately manipulating stats, but I think we are in a position to say the following:

Looking at the annex 5 last night I note footnote 2 which says he has, for the purpose of comparison, used stats for children known to social care in maintained schools from 2005 which include disabled children and are the latest available stats. I can see no reason until now for the Government to require disaggregated abuse returns that show how the child receives education.

This may indicate that the stats are unavailable for all schooled children (including independent schools) and there are no disaggregated stats for abuse that do not already include the target population (EHE) which he could have used, and certainly no new ones. I expect the stats he's using are the only ones that don't automatically include EHE children.

So, if we criticize him publicly he can reply that his hands were tied.
Then we would have to try to reply publicly, which we probably won't get the opportunity to do, saying he should have used his questionnaire to formulate disaggregated stats for both groups in order to even begin to address the question at the heart of the review, and he hasn't. He cannot merely assume that the two groups have the same rate of disability, for example. So his comparison is unreliable and inadmissible, as well as leaving the door open for a politically convenient misunderstanding. This is what he should have reported back to the Government. However, he is quite clear in the review (somewhere 8:12 and maybe elsewhere earlier) that the number of abuse cases is insignificant to his recommendation 22; that the potential for using EHE to cover abuse remains and should be policed for. Why bother having a review then?
We then get to the police state questions. Should we have one? How will we fund it? Will it protect or endanger children (by spreading resources too thinly)? Do the Government have the right to sacrifice our civil liberties for us? No, they don't, not without proportionate cause/risk.

Sally said...

David Law seems to hold the same position about need to regulate for potential for abuse even where abuse doesn't exist, or is significantly lower than general population.

Bev said...

My goodness, I feel so much better knowing that we have you on our side. Thank you... from all my family.

Maire said...

Sally,

You are right about Section 37. We initially had something about Section 37 but deleted it because it covers cases already gone to court. Whereas the text sought to highlight problems with the measure used.

Thanks for the info that Section 17 might include siblings, we didn't know that. Ditto about badly handled cases under Section 47.

We think your following up of the journos' is really important. If it was possible for their notes be used in a submission to the select committee as they would surely expose any dishonesty in the press release briefings. We have an Foi in for the people he or the department briefed and if we ever get an answer think it might be useful asking for their recollections of it. We will be putting an Foi in for the count of the answers to question 22 of the questionnaire.

My reading of the 2005 stats is that they aren't broken down by type of education - so the 3% figure for the population as a whole will include EHE children.

Agree absolutely that no one should acuse badman of manipulating the statistics. It's more that further details about the pros/cons of each figure was not spelt out so that a reasonable assessment of their value/usefulness can be made.

His LA questionnaire was a missed opportunity to collect useful data. He asks about use of HE as a cover for child abuse etc at Q22 (or 51 on the internet version) and asks about the number of Serious Case Reviews involing EHEs. But he didn't ask about non-EHE, without this as a comparator the responses he got will be of limited value.

Bruce and Maire

Maire said...

Thanks Elizabeth and back at you, I see your work in many places, and Bev, we are all doing what we can I know, and the whole is certainly greater than the sum of the parts!

Claire said...

I have also written to the UKSA. Would you mind contacting me to discuss?
(This is the first time I have ever commented on a blog so not sure if you will automatically receive my contact details. If not, you can find me on the BRAG forum as TheBladesFamily.)

Maire said...

Done Claire.

FEEDJIT Live Traffic Feed