Think-Tank Science

Posted by Pete Hague on 13 Jan 2012

This blog post is based on a report I made for the Pod Delusion - a skeptical podcast about interesting things.

I was recently made aware of a report, published by the think-tank Civitas claiming that wind turbines are useless at reducing carbon emissions. Like many people, I am highly suspicious of think tanks, and in this case it appears my suspicions may have been justified.

Before I go on, I should point out that I have no professional expertise in this area - but am concerned about climate change and the energy security of the UK , as I am sure most people are. It is an issue that affects all of us, so we should all try to keep informed.

Civitas didn’t bother to send their report for proper peer review, so I am going to have to review it for myself, without the benefit of it first being checked by experts in the field. Despite it being implied by their press release this is a scientific report, it kicks off with primarily economic arguments, and fairly poor ones at that. It simply states neoliberal positions (that any costs imposed on business “reduce competitiveness”) without even showing evidence of this, defining what they mean by competitiveness, or why its a good thing to pursue.

It also goes on, at some length, about the carbon emissions of other countries, notably China, as a way to argue that reduce the carbon emissions of the UK is futile. Not only is this a load of crap - its a transparent rationalisation for bad behaviour to say “everyone else is doing it” - but it is also entirely irrelevant to the question that the report is supposed to be addressing, which is the relative cost of various forms of energy.

When it finally gets to the meat of its argument, the general quality of the science doesn’t seem to me to be all that great. Essentially what the report does it take a previous studies of the relative costs of power generation, and then adds a bunch of cost to the values for wind power, and the calculation of those costs is of dubious accuracy. The report does pay lip service to the this with phrases such as ‘great uncertainties’ and ‘caveats’ - but doesn’t give quantitative errors in any of the numeric values it presents, and simply refers the reader to the report’s references.

And the references... part of the case of the supposed load balancing problems caused by wind power is made by referencing, in academic style, a BBC news article by a weather man, with a quote saying that cold weather is associated with high pressure, and high pressure with no wind. Having personal experienced cold, windy weather I find it hard to believe this is a hard and fast rule, and the authors of the report make no attempt to pin this down quantitatively.

What they do instead is provide a bar chart showing the energy provided from various sources, such as coal and wind, on a particular cold day - the 21st of December 2010. This doesn’t bode well.

The references of this report may contain some data - I will get to that later - but the data content of this report itself is one single cherry picked day.

The next plank of their argument rests on several assumptions, including one that the power demand of the UK will remain heavily biased towards the south whilst wind generation will remain biased towards the North. Are we to believe that, if energy supply becomes easier and thus cheaper in the North than the South, that industrial energy consumers won’t relocate? This seems to me to be a classic case of static analysis; extrapolating from existing trends without taking into account the complexity or responses of a system - a fallacy brilliantly satirized in a 2006 article in The Economist, predicted exponential growth in the number of blades on a razor based on a handful of data points.

Much of the reports analysis rests on the conclusions of another report by Colin Gibson, apparently also not peer reviewed, that contains such statements as:

As if, having pulled numbers out of thin air, using them to calculate a median suddenly makes it scientific! This error is repeated by placing numeric probabilities on the low, central and high estimates produced by Mott MacDonald - again without any justification. This data then produces a nice cumulative probability graph, in an attempt to again make it look like he isn’t just making numbers up.

Furthermore, the Mott Macdonald study, mentioned above, differentiates the prices of different energy sources based on “First of a kind” and “Nth of a kind” - and places offshore wind in the “First of a kind” block. This is used as a basis for some of the predictions about offshore wind - which would seem to show the age of the study, from which much of the data this report rests on comes - as we now have a substantial deployment of offshore wind in the UK since then. It seems to me the categorisation of it as “First of a kind” is rather tenuous at this point.

All this works towards producing the main figure used to demonstrate their point:

Despite the shaky basis for these projections, there is no indication in this figure of any uncertainty. No error bars, no low medium and high estimates. Just a single value for each energy source, stated as if it were absolutely 100% certain.

There is a very noticeable progression in confidence from the references of the report, through the report itself, and the press release. The references admit to making numbers up to get a rough estimate; the report uses these numbers, simply states there are ‘caveats’ to them, and them presents them in text and in graphics without indicating any possible error in them. The press release states the conclusions derived from these numbers with absolute, cast iron certainty. At no point in this progression is there any real peer review, as far as I can tell.

And that is the big problem; I am not an expert in any of the disciplines this report covers. I’m not a meteorologist, or a climatologist, or an engineer. I can spot blatant violations of the scientific method, but the fact is that if this is being presented as science it should have been sent through a peer reviewed journal for proper evaluation before Civitas starting throwing around press releases.

Others have uncovered more specific technical issues with their report, but as far as I am concerned, what I’ve spotted here in itself is enough to seriously question the validity of this report and its conclusions.

I have not had the time to give this report as thorough criticism as it deserves, but again that is the problem - non-specialists should not have to delve as deeply into the references as I have done in order to assess a piece of science publishing that is not in their field of expertise, but is nevertheless relevant to their daily lives.

I may be wrong in my unflattering assessment of this report, but it is up to the authors of the report, or anyone who wishes to support them, to demonstrate the point they are trying to make, and as far as I am concerned what they have produced does not at all support the conclusions they draw.

0 Comments