I was beginning to write a final post on this issue and apply Occam’s razor based on the information I had gathered and based on the GOP not having (at that time) contacting Sutherland to ask them to clarify how they obtained the email list. I was going to reject the option of Sutherland gathering a list themselves from various sources given the questions it raises about their sampling efforts – it simply didn’t make sense. It turns out that would have been the wrong thing to do and Occam’s razor would have failed in that instance.
First, based on information passed on to me, it appears the GOP may well have finally contacted Sutherland and asked them to clarify how they got the delegate email lists. It appears the study lead was flustered when specifically asked how he got the email addresses and erroneously stated they came from the party. He confused the fact that they had a delegate list from the GOP which did not include email addresses. Sutherland then contracted with an outside group(s) and use internal resources to harvest email addresses from the internet. I don’t know why they wanted incur this expensive and tedious process rather than just calling the delegates and surveying them by phone (also providing for better statistical controls and sampling methods) but it’s their money/time to spend. If they were gathering emails in order to intentionally skew survey results (see below re: PETA etc), it again seems to be a tedious and spendy way to achieve such. Meh, who knows.
The problem with harvesting email addresses online is that people begin to ask how xyz got their email address. It’s also what spammers* do (with less focus) and it annoys people. Thus, delegates began asking how Sutherland got their email address and were left to believe Sutherland was given the list based on calls and the language in the Sutherland’s survey results publication.
As I’ve previously noted, Sutherland’s report has statistical issues. Email surveys tend to be inaccurate, lacking sufficient statistical controls. Just as important, however, is that surveyed sponsored by special interest groups tend to skew the results as recipients will be much more prone to respond if they agree with the sponsor than if they disagree – a PETA survey will have a higher response rate among members and like-minded recipients versus hunters and barbecue enthusiasts. The problem is only compounded with an incomplete list of the survey population and email harvesting is a fairly good way to end up with an incomplete and/or inaccurate survey population list. Use of internal databases or affiliate organizations lists to gather survey recipients results in even more skewing (bias) as they, predominantly, consist of respondends who agree with the organization (eg: finding email addresses from a PETA database while others have to be found by harvesting is going to lead to a higher proportion than true of PETA-affiliated respondents).
I don’t know how often Sutherland has applied such methodology to their surveys but this demonstrates the necessity of analyzing surveys (especially the ‘internals’) before drawing conclusions or accepting statements. Various special interest groups on all sides may choose to skew results and it harms their credibility – you don’t know which results are valid vs cooked. At this point, I don’t know which Sutherland studies I dare trust. Ergo the adage: “Lies, damned lies, and statistics”. Caveat emptor, indeed.
*Note: Unsolicited email is known as “spam” but I believe political activities are exempt from spam law and Sutherland may qualify under this exemption. Imagine that – politicians are not subject to their own legislation.
UPDATE (12/14): Thought this was of interest. Obama’s campaign is harvesting Republican email addresses to send unsolicited e-mail (spam). Similarly, it appears, Sutherland harvested email addresses to send unsolicited email. Ed Morrisey provides some pertinent background and advice:
It’s almost certainly not a violation of the CAN-SPAM law, since campaign advertising isn’t “commercial” in the sense of the law’s definition, but being a spammer isn’t going to increase the credibility of Team Obama, either. Basically, as Sean Hackbarth pointed out on Twitter earlier, if you get unwanted e-mail from political campaigns, either unsubscribe or mark it as spam — which it truly is. And choose your friends a little more wisely in the future, too.