Jun 15, 2010

The Activists, the Poll & the Data

"Global warming concerns rising in US" declared a headline on the website of the Canadian Broadcasting Corporation (CBC) last Thursday. If true, this in an important development. But the first clue that something may be amiss is provided by the fact that the news story lacks a byline (it isn't attributed to a particular journalist). This suggests it was written up quickly by a junior employee or by an editor-in-a-rush and was based on nothing more than a press release.

For a disturbing explanation of why journalism of this sort has become commonplace see the 2009 book Flat Earth News. The first few chapters describe how news stories having virtually no basis in reality get promulgated far and wide by the media and how this is a predictable result of recent developments in the modern newsroom. Given that the average "churnalist" is expected to produce increasing amounts of copy, a reporter is virtually guaranteed to take a press release at face value, regurgitate it in different words, and then move on to the next story.

The CBC article mentioned above has attracted 350 reader comments. Assuming that one reader in a thousand leaves a comment, potentially hundreds of thousands of people now think the (internationally distributed) downturn in concern about global warming has halted in America. But has it?

ACTIVISTS ARE US
It's worth noting that the public opinion poll being cited was written, paid for, and analyzed not by a disinterested polling firm that routinely monitors public attitudes on a variety of topics. Rather, the four principle investigators are associated with the Yale Project on Climate Change Communication and the Center for Climate Change Communication at George Mason University.

Quick translation: these folks have agendas the size of the Sahara. The Yale outfit tells us its "mission" isn't to study climate change attitudes in a dispassionate manner but to:
Catalyze action by the general public and leaders of government, business, academia, and the media... [bold added]
Furthermore, we're told the Yale project was established after a 2005 conference that "came together to develop an action plan to engage American society on climate change."

All of this makes it clear their purpose is not primarily scholarly. Instead, the Yale Project on Climate Change Communication is an activist organization.

The same is true for George Mason University's Center for Climate Change Communication. It says its mission is to "conduct unbiased public engagement research" - which sounds great until one reads the end of that sentence:

..so that collectively, we can stabilize our planet's life sustaining climate.
Check out the academic bio of one of the George Mason researchers involved in this particular poll:
Connie Roser-Renouf has a long-standing interest in the use of mass media to facilitate social change...She now directs her attention entirely to climate change, believing it to be the largest challenge humans have ever faced. [bold added]
And here's the academic bio of another researcher, Edward Maibach (who happens to be the founder and director of the George Mason center):

In 2006, while on a walk in the mountains...Ed had an epiphany that forever changed his life. He realized that climate change is the ultimate threat to the public’s health and wellbeing...Ed’s research interest is focused on the question: How can we use communication and marketing to influence the behavior of populations for the benefit of society? [bold added]

Some days I feel utterly naive. My mental image of American universities as places in which students become equipped to sort credible arguments from nonsense is hopelessly out-dated. Instead, it seems universities now employ self-important activists who get paid handsomely to discover which marketing techniques will advance their personal politics.

But putting that aside, let's look at the numbers. It so happens I know more than the average person about public opinion surveys because my husband, prior to leaving the industry, worked in market research/polling for two decades.

TIMING MATTERS
The timing of a poll can be important. The most recent climate change numbers were gathered during the final two weeks of May 2010. But those numbers are being compared to ones gathered between December 24th 2009 and January 3rd 2010.

Yep, the previous data was collected smack dab in the middle of the holiday season. Fortune 100 companies - who need reliable data because multi-million dollar decisions hinge on it - would never permit a survey to be conducted at that time of year. The risk of data wobble connected solely to people's seasonal frame of mind is considered too high.

Moreover, industry best practice is to conduct ongoing surveys at the same time each year. Thus, if the researchers in this case want to feel certain their data isn't being contaminated by seasonal variability, they should schedule the next wave for the last two weeks of May of 2011.

Instead, we have two waves of data collected five-and-a-half months apart. In both cases the margin of error is plus or minus three percent.

So when a press release says that the view that global warming "is caused mostly by human activities rose three points" and that the "number of Americans who worry about global warming rose three points" these results are meaningless. Because this falls within the margin of error we can't be sure anything has really changed.

The press release also tells us that "public belief that global warming is happening rose four points" and that "the number of Americans who said that the issue is personally important to them rose five points."

Once we subtract the margin of error, we have movement of one percent and two percent on these two questions. This is thin ice indeed on which to base a news story that begins with the line: "Global warming is once again becoming a hot topic for Americans."

THE BOTTOM LINE
According to this survey data, there has been no change in public opinion worth talking about. The numbers may well shift decisively in one direction or another at some point in the future, but it is speculating wildly to say that this has already occurred. The insitutions of higher learning that sent this rubbish to the media should be ashamed of themselves.

That being said, had the CBC journalist taken the time to examine the data directly, it seems to me he would have discovered some newsworthy tidbits. For example:

  • Even though the BP oil rig explosion killed 11 people on April 20th and oil had been gushing into the Gulf of Mexico for nearly a month prior to the start of the survey (May 14 to June 1), 62 percent of Americans nevertheless still supported the expansion of offshore drilling (pp. 5 and 7 of this PDF).

  • 65 percent of respondents said the evironment should be protected even if it reduces economic growth and 77 percent said they supported regulating carbon dioxide. While at first blush this sounds like a mandate for vigorous government action, it must also be noted that 65 percent of respondents were opposed to a mere 25-cent-per-gallon increase in the gasoline tax even if this money were returned to them via a reduction in federal income tax. Moreover, 45 percent were opposed to paying an extra $2.50 a month for electricity. (pp. 5-8 of this PDF).
Although many of the survey respondents seem unclear about the connection between the two, there is no way to dramatically reduce/regulate emissions without increasing the cost of gas and electricity. When the price of energy goes up, the price of everything that is grown, manufactured, or shipped increases and those near the bottom of the income ladder experience genuine hardship.

The public is profoundly schizophrenic about these matters. We like to declare that the environment is important to us, but we aren't prepared to spend much of our own money on it. This is why support for environmental policies is often described as being a mile wide and an inch deep.

FUN WITH NUMBERS
Since it's obvious that the numbers which made it into the press release were cherry-picked, let's indulge in some cherry-picking of our own. Keeping in mind the three percent margin of error here are some other revelations:

  • the percentage of respondents who were "exremely sure" that global warming is happening fell from 24 to 20 (p. 2 of this PDF)

  • the percentage who are "somewhat sure" that global warming is not happening rose from 34 to 44 (p. 2)

  • those who think there's "a lot of disagreement among scientists about whether or not global warming is happening" increased from 40 to 45 percent (p. 3)

  • the percentage who think global warming will cause them "only a little" personal harm rose from 22 to 27 percent (p. 4)

  • those who think global warming will harm US citizens "a great deal" dropped from 22 to 16 percent (p. 4)

  • those who think "global warming will harm plant and animal species" a "great deal" fell from 43 to 40 percent (p. 5)

Then there are my two favourite cherries. Last December, just prior to some of this data being collected, the two-week Copenhagen climate extravaganza took place. In the months leading up to that summit celebrities and green groups uged us to be alarmed and the mass media got hysterical. On the day the summit began 56 newspapers worldwide ran an editorial (many on the front page) in which the event was characterized as "Fourteen days to seal history's judgment on this generation." Television, radio, and news-oriented websites all joined the chorus.

Yet when respndents were asked mere days later, over the holiday season: "How much had you thought about global warming before today?" only 15 percent said "a lot." Combined, more than half (54 percent) said they'd thought about global warming only "a little" or "not at all" (p. 6).

Even more interesting, when they were asked: "How important is the issue of global warming to you personally?" only five percent said it was "extremely important" (p. 6).

No wonder the researchers who wrote this survey think they need to employ marketing techniques to communicate their personal sense of urgency. The fact of the matter is that global warming is not a pressing issue for the vast majority of Americans.

MORE DATA CONCERNS
Anyone who takes the time to look closely at this survey may find further reason for concern. The researchers claim the "results come from nationally representative surveys of American adults, aged 18 and older." But when those folks were asked in May 2010 for their political party affiliation 42 percent said they were Democrat, 25 percent said they were Republican, and 24 percent said they were Independents. (p. 9 of this PDF)

Gallup asks Americans this exact question each month. Data stretching back to January 2004 appears here. During the same time period (May 24-25) that the academic researchers were quizzing respondents about their political affiliation Gallup compiled somewhat different numbers:

  • Democrats 30% (vs 42% by the academics)

  • Republicans 28% (vs 25%)

  • Independent 40% (vs 24%)
According to Gallup, over the past year between 29 and 37 percent of the population described themselves as Democrats (in Apr. 2010 and July 2009 respectively). Yet 42 percent of those polled in the academics' survey were Democrats.

Similarly, over the past year Gallup says Independents comprised between 33 and 42 percent of voters. According to Gallup's data, at no time during the period January 2004 to the present have independents been less than 27 percent of the population. Yet only 24 percent of the respondents in the academic survey were Independents.

It therefore appears that Democracts were over-represented and that Independents were under-represented in the sample from which the academics derived their numbers. On a topic as polarized as climate change, this could significantly bias the results.

There's also a problem with lack of disclosure. I sent e-mail to all four of the academic researchers last Friday morning asking to see the entire questionnaire since certain survey questions are missing from the information released by the academics. This is important because it's well known that the answers people provide during a survey can be influenced by the ones they were asked immediately beforehand.

One of the researchers, Anthony Leiserowitz, responded promptly and politely. Since he cc'd the others in his response and none of them has been in touch, it appears they are all comfortable with the fact that he declined to provide the missing questions. His e-mail reads in part:

We cannot release this entire questionnaire yet, as we are in the process of preparing several additional public reports, as well as papers for publication in scientific journals. [bold added]

Where have we heard that before? Here are academics eager to influence the public debate about climate change. They issue a press release about non-peer-reviewed research findings hoping to attract media attention (see here for another reporter's coverage). But when an informed observor requests a full list of the questions so she can satisfy herself that the survey was conducted in a fair manner, the academics feel no obligation to provide such information. In other words, like other climate researchers, they expect the rest of us to blindly trust them.

Doesn't it make you feel warm and fuzzy all over to hear that this research is destined to appear in scientific journals? Despite its shortcomings perhaps it'll get cited by the Intergovernmental Panel on Climate Change.

..

>> The media, Mother Nature & oil spills
>> BP, Greenpeace & the big oil jackpot
>> Shielding climate orthodoxy from free speech
>> Sudan fights climate change for the IPCC