I am sure that you have all heard the word consensus used many times with regard to what is often called Anthropogenic Global Warming (AGW), or more recently, Global Climate Change (GCC). However, over the past year or so there has been another phrase bandied about to support this so-called consensus, that “97% of climate scientists agree with the consensus view that humans are the primary cause of global warming” (or something to that effect).
The source of this second claim (that attempts to reinforce the first) is from a 2013 paper by lead author John Cook, a solar physicist who operates the website Skeptical Science. The paper in question, Quantifying the Consensus on Anthropogenic Global Warming in the Scientific Literature, or simply Cook et. al. (2013), concludes, “Among papers expressing a position on AGW, an overwhelming percentage (97.2% based on self-ratings, 97.1% based on abstract ratings) endorses the scientific consensus on AGW”.
That statement looks fine and dandy at first glance and it seems to support the data as shown in the Cook et. al. paper. However, there is far more to this statement than meets the eye.
Methodology is Everything
In order to undertake their study, Cook and his co-authors searched the ISI Web of Science looking for papers published from 1991-2011 using the search terms “global warming” and “global climate change”, and they further restricted their search to articles only (which excluded books, discussions, proceeding papers, and other types). This yielded a total of 12,465 papers, which were then rated into types according to Table 1 below, the result of which was the elimination of papers that were not peer-reviewed, not climate related, or without an abstract; this resulted in a total of 11,944 papers from 29,083 authors in 1980 journals.
The resulting 11,944 papers where then sorted into seven levels according to their level of endorsement of AGW as shown in Table 2 below. We can see that amongst the seven levels in Table 2, that they can further be reduced to 3 primary categories, two of which (endorse/reject) have their own three levels of relative endorsement/rejection. The primary categories are: Endorse AGW, No Position/Uncertain, and Reject AGW.
Cook et. al. also performed a self-rating analysis, whereby they emailed 8547 authors with an invitation to rate their own papers. They received 1200 responses (a 14% response rate), and a total of 2142 papers received a self-rating from 1189 authors.
While the self-rating section of this study is quite interesting, I’ll be limiting my analysis of the claim that “97% of climate scientists agree with the consensus” (paraphrase) to only the first part of the study, that which deals with the papers themselves. The reason for this decision of mine is two-fold: first, due to the very small number of respondents who self-rated, there could be the inclusion of bias toward a particular position; and two, I do not have access to the data file for the self-rating portion of the study (I only have the data file for the abstract analysis portion of the study).
It must be asked, “what exactly is the consensus in this study?”. Cook et. al. (2013) doesn’t define this term for us, so we have no choice but to infer its definition from our own opinions, right? Well, not exactly.
You see, there was another paper, Legates et. al. (2013), that served as a critical response to Cook et. al. (2013). While this paper is behind a pay wall and thus cannot be fully accessed by myself, it is fortuitous that Cook responded to it in another paper, Bedford and Cook (2013). In this particular paper co-authored by Cook we see the sentence, “Cook et al. (2013) found that over 97% endorsed the view that the Earth is warming up and human emissions of greenhouse gases are the main cause”.
So while in the first Cook paper we only get the statement, “Among papers expressing a position on AGW, an overwhelming percentage (97.2% based on self-ratings, 97.1% based on abstract ratings) endorses the scientific consensus on AGW”, we get a followup paper from Cook stating that “Cook et al. (2013) found that over 97% endorsed the view that the Earth is warming up and human emissions of greenhouse gases are the main cause” (emphasis mine).
It appears that we now have Cook’s definition of what the consensus is that he referred to in his first paper, and it is entirely consistent with the IPCC’s claims as laid out in their 2007 statement, as well. The consensus is apparently that humans are the main cause of global warming.
The only problem is that Cook is full of it. He’s being entirely dishonest!
A Little Look at the Numbers
As mentioned above, the methodology employed by Cook et. al. (2013) was to take the 11,944 papers and separate them into seven distinct levels of endorsement, and that these seven levels could further be separated into three primary categories as outlined in Table 2 above. The results were then compiled and shown as percentages in Table 3 below, though one can clearly see that level 4 was further divided between “uncertain on AGW” and “no AGW position” in the table.
As can be clearly seen, 32.6% of all abstracts endorse the AGW position, a majority of 66.4% having no position on AGW, and the other categories represent less than 3% of the total. However, we also see the 97.1% figure that is the topic of our inquiry, which represents the percentage of abstracts with a position on AGW who actually endorse (rather than reject) the AGW position.
This all seems consistent with Cook’s claims, right? No, not exactly.
If you’ll remember, the primary category of endorsement for AGW was divided between three distinct levels of relative endorsement (levels 1, 2, and 3 in Table 2). The first is level 1, “explicitly states that humans are the primary cause of recent global warming”. The next level 2 says, “explicitly states humans are causing global warming or refers to anthropogenic global warming/climate change as a known fact”. The last level 3 states, “implies humans are causing global warming”. So only level 1 states that humans are the primary or main cause of global warming, while level 2 merely states that humans “are causing” global warming (which doesn’t tell us how much they are causing it), and level 3 merely implies that humans are causing global warming (which again doesn’t tell us how much they are causing it).
So the 97.1% figure that is the topic of our enquiry is actually the sum of levels 1, 2, and 3. If this is the case, then it certainly cannot be said that the statement from Bedford and Cook (2013) that, “Cook et al. (2013) found that over 97% endorsed the view that the Earth is warming up and human emissions of greenhouse gases are the main cause”, is at all accurate. After all, some of those papers representing that 97.1% in Cook et. al. (2013) must have fallen into both levels 2 and 3, and not just level 1 (which is the only level that states that humans are the primary or main cause of global warming). Nor can it be said that the statement in Cook et. al (2013) that, “among papers expressing a position on AGW, an overwhelming percentage (97.2% based on self-ratings, 97.1% based on abstract ratings) endorses the scientific consensus on AGW”, is accurate, because the subsequent paper by Cook defines the consensus in this statement as being the position that humans are the main cause of global warming.
And how do I know that not all of the papers that represent that 97.1% figure fall into only category 1? Because logic would dictate as much, but also because I have the data file for Cook et. al. (2013). Here’s what the breakdown of each level looks like:
Level 1 = 64
Level 2 = 922
Level 3 = 2910
Level 4 = 7970
Level 5 = 54
Level 6 = 15
Level 7 = 9
Clearly, we can see that there were more papers that rejected AGW than there were that supported the notion that humans are the primary or main cause of global warming (i.e. level 1). Further, when we break down the three levels that represent the category of endorsement of AGW, this is what they look like as a percentage:
Level 1 = 1.6%
Level 2 = 23%
Level 3 = 72%
I’m not a climate scientist, but the simple fact is that based upon the evidence in his own papers, John Cook is misrepresenting his own study and is fabricating the idea–based upon his own definition and that of others–that 97% of climate scientists endorse the consensus that humans are the primary or main cause of global warming. What is worse is that I’ve heard this statement “97% support the consensus” from all sorts of sources. From the ignorant masses on the internet, the talking heads on television, in news articles, and even from the President of the United States himself. While it could quite probably be the case that they’re simply repeating what they’ve heard others say, it is still entirely irresponsible to make such a claim as fact without checking it out for yourself.
I’ve done the homework for you. The next time that you hear somebody make claims about a consensus, you’d be entirely correct to point out to them that 1.6% does not represent a consensus.