Unsound evidence wastes millions
If a bridge falls down, something has obviously gone wrong. But if a piece of social survey research such as a Housing Needs Survey (HNS) is badly flawed nothing is obvious. The consequences may be just as serious: the community involved does not get the affordable housing it needs, and public money is wasted. But there is normally no visible crash.
A few weeks ago in Redditch a HNS did crash. As can be seen from the attached quotes, the Council acknowledged that their David Couttie Associates (DCA) HNS was unsound. How could it happen? After all, HNS have been around for a decade or more. The answer is: no quality control. When assessing tenders for HNS, councils often go for the cheapest. They assume that if the consultant has done a few HNS (and DCA has done around 170) they must be all right. Councils often use points systems to evaluate tenders, say 60% on price and 40% on quality. These can lose sight of the wood for the trees.
Some councils, by careful examination of the material, have been able to understand the situation. Most have not. Councils should not generally be blamed. HNS involve a highly specialised statistical expertise. Few if any councils have the capacity to properly assess this. One of DCA's main defences, when we began our campaign of objection at inquiries, was that nobody had ever complained about their work. This suggests the inability of many councils to make such specialised assessment, and of course other parties (mainly housebuilders) actually profit from poor HNS: it costs them less to provide inappropriate affordable housing.
This disaster has been foreseeable for a long time. In 1995 Fordham Research published, with what is now Room, a booklet on how to do HNS. In the introduction I wrote:
'It may seem absurd to suggest that the millions of pounds spent on HNS have not produced estimates of housing need. Unfortunately it is true'.
I called on (what is now) the ODPM to get the Audit Commission to install quality control. Nothing happened.
We clearly have a professional interest in ensuring that the whole field of social survey to support housing and planning policy is respected. We would like to feel that the quality of all HNS is high and that public money is not wasted. We were driven to take direct action, after a decade of warnings, by the fact that we were about the only organisation with the specialised knowledge that would enable proper quality testing. Hence in 2003 we started appearing at Inquiries to provide evidence, where we believed that poor work had been done.
The immediate reason for beginning the resultant campaign is that GOSE advised us that we should object to HNS at Local Plan/UDP inquiries. We started in 2004 to object to HNS which we believed to be unsound. We have appeared at about a dozen. It was a last resort. We have visited ODPM on several occasions to try to persuade them to act, with no outcome. The Redditch result may change this. The Borough's letter states baldly that
'The David Couttie Associates HNS is not a clear, rigorous or realistic assessment... [it does not conform to ODPM Guidance and] it cannot properly be used as the basis for local plan policy'.
In other words the £40k or so spent on the Redditch HNS, together with the staff time input, was wasted.
The implications of Redditch's letter are substantial. The Redditch HNS was carried out in Spring and Summer 2005, and is DCA's 'latest technology'. It corrected a number of longstanding errors which had been pointed out by Fordham Research at previous LPI's. However the HNS was still not robust. Upwards of 100 DCA HNS are still current. As evidence of housing need, none of these is likely to be acceptable. It does not matter whether they are in places like Redditch or ones that are rural or big cities: the technical approach was very similar, and clearly it was flawed.
Furthermore, the DCA questionnaires lack certain key data for a rigorous assessment. Hence the situation cannot be rectified simply by re-analysis of the existing data. New data will have to be collected. It is not just that firm's HNS which are unsound: many current HNS 'brands' are not. Around half the councils in Britain may be using HNS evidence which is not 'fit for purpose'. That's about £8m just to put things right. That is only the tip of the iceberg: many more millions have been spent downstream on inappropriate affordable housing.
This bridge disaster could easily have been avoided. The evidence has been there for a dozen years. It is to be hoped that ODPM will ensure that a quality control unit is urgently established to check existing and future HNS. This should at least prevent further waste of public money.
Dr Richard Fordham
Oct 05