Thursday, February 14, 2008

Criticism of the "Hockey Stick" #1: Data Mining & A “New” Statistical Method

This will be my first post of hopefully a series of posts criticizing Michael Mann and the MBH98 dataset. His millennial reconstruction is the basis for global warming alarmism, and it has been featured in the 1995, 2001, and 2007 U.N. Intergovernmental Panel on Climate Change reports. Both his flawed statistical methodology and his reliance on unreliable tree-ring proxy make his “hockey stick” reconstruction illegitimate. Many of my arguments are based off of the publications of Steve McIntyre (a data analyst) and Ross McKitrick (an environmental economist), both of whom have been extremely critical of Mann and his colleagues.

McIntyre and McKitrick have been fully endorsed by the Wegman Panel Report (headed by Edward Wegman of George Mason University and the Chairman of the National Academy of Science Committee on Theoretical and Applied Statistics), have testified before Congress, have significantly influenced the National Research Council Report on the issue, have presented to the National Academy of Sciences, has have been published in the Geophysical Research Letters, Energy and Environment, and Nature. McIntyre runs a weblog ( in which he uses the statistical skills that he used in the private, mine prospecting sector to analyze claims made in climate science from both sides of the debate. I read it every day, and I find his work to be as unbiased and professional as it is illuminating. In 2007, the site tied for the “Best Science Blog” in the weblog awards in 2007.

In my last post, I wrote three reasons why temperature reconstructions over the past millennium are relevant to the modern global warming debate. For ease of reference, here they are again:

1. It would shed light on the idea that today’s temperatures are out of the natural variability.2. It would possibly correlate temperature with societal development.3. By establishing the behavior of the curve, we would gain greater insight into the strength of solar intensity as a climate modifier. Specifically, if temperatures followed the same trend as solar intensity, then we could suggest that perhaps solar intensity has a dominant role in climate change. Yet, we have to be careful; this is merely speculation. To actually make a scientific case for a strong solar sensitivity, we would have to establish the mechanics by which such a process would work as well as the historical correlation.
Now, the most well-known and publicized reconstruction is often described as the “hockey stick” chart, and those who support it (specifically, authors of Real Climate) are dubbed the “hockey team.” Specifically, the reconstruction is MBH98 (and more recent variations) and the lead scientist is Dr. Michael Mann of Penn State. Here’s what his weblog, Real Climate, says about him:

Dr. Michael E. Mann is a member of the Penn State University faculty, holding joint positions in the Departments of Meteorology and Geosciences, and the Earth and Environmental Systems Institute (ESSI). He is also director of the Penn State Earth System Science Center (ESSC).

While I was relatively balanced in my description of paleoclimate issues in the last post, things will begin to change, especially concern Mann and the hockey stick, for it is an issue that I am especially passionate about. Michael Mann’s team has successfully infiltrated television, magazines, textbooks, movies, and popular society with his reconstruction that is filled with deceitful statistical tricks and unreliable tree ring proxies.

The result is a chart with a slightly negative temperature trend until mid-19th century, at which point there is a sharp uptick in global temperatures.
So let’s go back to the three reasons millennial paleoclimate is important.
1. It suggests that modern temperatures are far from natural variability.
2. Any perceived societal development during the Medieval Warm Period would have been unrelated to climate, which shows no upward trend in temperature during that time.
3. Temperature trends seem to closely follow co2 trends, while solar trends seem completely irrelevant. This implies that atmospheric greenhouse gas concentration is the most important climate forcing, which would suggest that modern waming is indeed due to changes in co2, not total solar irradiance (TSI).
For reference, here are three graphs: the Hockey Stick temperature reconstruction, TSI (measured by changes in atmospheric C14 levels), and CO2 concentration.

Now that I have given a brief introduction to the issue, it’s time to start exposing the manipulation that Mann did to achieve the desired reconstruction. This post will look specifically at what Mann described as a “new statistical approach.” What was “new” about it was the way he used Principle Component Analysis (PCA) to mine for the hockey stick shape. In order to effectively draw out trends and use data, the large collection of data from specific proxy reconstructions is condensed into a single series (y). This series takes the form
y = a1x1 + a2x2 + . . . + akxk,
where xn is a single data set, and an is the weight assigned to that data set.

To determine the average of the data, the weight for each data set would be 1/k, so that the entire set would add up to equal 1, but because Mann used PCA, the weights were given different values, dependent upon the variation of the datasets from a given value. This means that the graph of the series will be significantly altered by datasets with large variance, while datasets with low variance will not be very relevant to the end product. This series, y, is referred to as the first Principal Component, PC1. After PC1, there is still some unexplained variance from datasets with less variance. So, PCA also yields PC2, which describes the second Principal Component. This pattern continues until the PC is close to zero. Thus, PCA is an important way to identify variance in a set of data. For PCA to work properly, the data has to be standardized (given a mean of 0 and a variance of 1).

So how did Mann take advantage of this perfectly legitimate method for statistical analysis? To determine the variance, one must determine the mean, and instead of determining the mean for the period under investigation (1400-1980), he applied to mean of the period 1901-1980 to determine the variance. This shouldn’t have caused a lot of issues, for in most of the North American tree ring group, there is not significant variation in temperatures from 1980 back to 1400. Well, actually that’s not completely true, so I need to explain exactly what I mean before I continue. A proxy is defined as an indirect reading of something, in this case - temperature.

When the dendrochronologist reads the width of tree rings, it does not tell him or her the exact temperature of that year. Instead, it gives a width that then can be compared to other years, specifically ones that we know the temperature of through modern instrumental data collection. Through that process, the magnitude of changes in temperaure is established. Therefore, although the temperature magnitude of the changes is large, the departures from the mean over extended periods of time are much smaller, and that is what I mean by my statement that there is no significant variation in temperatures back to 1400 in most of the tree ring proxies.
And although most proxies show no variation (in comparison to previous centuries) beginning in the 1800s, a select few do - specifically, Bristlecone Pine tree rings. Because the mean year to determine variance was set in the 1900s, pre-industrial revolution temperatures are given higher weights, and thus, those Bristlecone Pines are given a tremendous sway in the reconstruction graph. There are significant problems that I will comment on in a later post concerning the use Bristlecones as a proxy, but in this current argument, I’m only trying to show that an undue amount of emphasis was added to these select few proxies from the Southwest USA.

To give an example, the Graybill-Idso Bristlecone proxy was given 390 times more weight than a normal, non-Bristlecone proxy from Mayberry Slough, AR through Mann’s “new” statistical approach. While normal PCA would allow the Graybill-Idso Bristlecone proxies only 8% of the variance, Mann’s technique allows the proxies account for almost 40% of the variance. It is in this manner that Mann et al. essentially “mine” for a hockey stick.

There will be more criticisms to come.

For the papers from which I take my information, visit McIntyre and McKitrick’s project page at