Some interesting remarks from a political scientist specializing in research on voter behavior - to keep in mind when watching the brainless pundits.
He's one of the political scientists that made the prediction that a generic Democrat, given the state of they economy in the last year and Bush's approval ratings should win by about 4-8%. Furthermore, those two pieces of data pretty well predict every presidential election result going back to the 1940s. So all of the daily campaign noise, from a political science perspective, is mostly pointless (shocker - Joe the Plumber didn't actually matter!); polls almost invariably collapse around the "fundamentals" (i.e. economic performance and incumbent approval rating) around October, which happened in this case as well. At any rate, this theory won me some serious cash on intrade last night, so I'm pretty happy with it.
But I still think Obama had a higher hurdle to overcome because of his foreignness, especially to older people. And he pulled that off remarkably well. A victory for the ages, for sure...
----------
http://redbluerichpoor.com/blog/?p=206
Election 2008: what really happened
November 5th, 2008
After a quick look at the election results and exit polls (from www.cnn.com), some thoughts:
1. The election was pretty close. Obama won by about 5% of the vote, consistent with the latest polls and consistent with his forecast vote based on forecasts based on the economy.
2. As with previous Republican candidates, McCain did better among the rich than the poor:
But the pattern has changed among the highest-income categories:
3. The gap between young and old has increased–a lot:
But there was no massive turnout among young voters. According to the exit polls, 18% of the voters this time were under 30, as compared to 17% of voters in 2004. (By comparison, 22% of voting-age Americans are under 30.)
4. By ethnicity: Barack Obama won 96% of African Americans, 68% of Latinos, 64% of Asians, and 44% of whites. In 2004, Kerry won 89% of African Americans, 55% of Latinos, 56% of Asians, and 41% of whites. So Obama gained the most among ethnic minorities.
5. The red/blue map was not redrawn; it was more of a national partisan swing. See this state-by-state scatterplot of Obama vote in 2008 vs. Kerry vote in 2004:
The standard deviation of the state swings (excluding D.C. and the unusual case of Hawaii) was 3.3%. That is, after accounting for the national swing in Obama’s favor, most of the states were within 3% of where they were, compared to their relative positions in 2004.
By comparison, here’s the 2000/2004 graph:
The standard deviation of these state swings was 2.4%. This was even less variation–2004 was basically a replay of 2000–still, the relative state swings of 3.3% in 2008 were not large by historical standards.
Again, Obama didn’t redraw the map; he shifted the map over in his favor. (Or, to put it more precisely, the economy shifted the map over in the Democrats’ favor and Obama took advantage of this.)
Here’s the map showing where Obama and McCain did better or worse than expected based on 2004:
6. Finally, how did the pre-election polls do? Unsurprisingly, they pretty much nailed the national vote. And what about the relative positions of the states? The pre-election polls did well there too, at least using Nate Silver’s aggregations. Here’s the scatterplot:
Pretty damn good. The standard deviation of the discrepancies, again excluding D.C. and Hawaii, is 2.5%, which is a big improvement on the 3.3% using Kerry04 alone.
I see some systematic patterns: Obama underperformed where the polls had him way down, and he outperformed where the polls had him up. We should go back and look at these patterns from earlier elections and see if this is consistent. If so, it suggests a way to improve forecasts for next time.
P.S. Age graph fixed from first posting; thanks to Andy Guess for pointing out the error.
To follow that up, some more interesting analysis from the same website. It turns out that if you also control for incumbent's approval ratings (in addition to economic data), the predictions get very accurate.
-------------------
The 3rd quarter data posted by the BEA yesterday are so dire that I [Hibbs] changed my tune and generated a new forecast based on the latest data.
October 31, 2008 update of Presidential Vote Forecast: Preliminary estimates of 2008q3 national income data released on October 30 by the BEA indicate that the economy has weakened so much that I have updated my 2008 election forecast: The Bread and Peace model now predicts a 2-party vote share for McCain of 46.25%, implying Obama will win by a margin of 7.5 percentage points.
Just as a reminder, here’s the graph of Hibbs’s model applied to earlier elections:
As you can see, the incumbent party sometimes loses but they never have gotten really slaughtered. In periods of low economic growth, the incumbent party can lose, but a 53-47 margin would be typical; you wouldn’t expect the challenger to get much more than that. Such things can happen (see, for example, Eisenhower’s performance against Stevenson in 1952) but it wouldn’t be expected.
---------
Thanks for the data, A. What a wonderful world we live in where a political scientist living overseas can make money on the Internet betting on his country's election results! Heck, Obama was definitely a safer bet than the Dallas Cowboys.
Heh, if it's true that fundamentals mostly dictate election results, why can't we dispense with the 2 years of campaigning nonsense? $1B spent on robo-calls, ads, conventions, private jets flying across the country, junk mail, rallies, etc. The two things that a campaign can't control for in an open-seat election, the incumbent and the economy, seem to matter the most, so can't the voters just decide on their own without being so harassed and bombarded at every turn? Well I guess that's why the world makes more sense in academia, but in practice it's chaos out there.
But still, the "human element" (campaign message, personality, celebrity, etc.) must matter, though it could be difficult to quantify in a model. An "un-named Dem" was projected to beat McCain, but if the Dem nominee was John Edwards or Al Sharpton, and controlling for all other variables, do you think they would be victorious today? Or if it was revealed that Obama had an affair, or terrorists attacked a major US city in October - do they factor in those "bombshells" on Intrade too?
I'm surprised that there was no "massive turnout among young voters". What was all that talk about energizing the youth and getting so many people to vote for the first time? What about the Dems registering 100,000's more new voters than the GOP? And the AP said that black voting wasn't higher than usual. So what was the Obama effect? Obviously he earned 7% more of the black vote than Kerry, but that demographic already swings so heavily for the Dems. He did seem to win >10% more of the under-30 vote than Kerry, but nationwide that only accounts for a roughly 1.5-2% gain in the popular vote.
Well, I guess we can be sure that scholars will be pouring over the nuts and bolts of this historic election for years to come. I wonder what lessons future campaigns will take away from 2008 and what new trends have begun. I guess besides TV, the Web is the key battleground for 21st Century politicians, so get used to seeing more campaign ads when you play Madden and such!
--------
Yeah, I sort of feel that as an academic, armed with these theories, if I'm not willing to put my money where my mouth is there isn't a lot of point. If these theories aren't right at least 51% of the time, might as well pack it in.
I suspect the answer to why we waste all this money on campaigns if the results are already more or less determined by fundamental factors is that:
1) spending probably matters on the margins. The general range of possible outcomes is already prefigured by the economic situation and incumbent popularity. But an extra million organizing the GOTV in South Florida in 2000 would have make a pretty big difference on the outcome.
2) You probably have to appear to be trying; if your opponent is going all out and you're being half-hearted in campaigning, then I doubt voters will take well to that. So it's a prisoner's dilemma; both candidates would be better off if they didn't have to raise so much money, but given that one of them starts raising it, the other has to appear to be putting in the effort too. That's why public financing of campaigns is probably a good idea; the coordination problem of credibly agreeing to not raise funds is solved by the government.
With regard to the human element, it's hard to say. I tend to think that people that follow politics closely do it like sports fans; you and I are pretty unlikely to change our opinions about which side to support (i.e. Raiders fans pretty much stay Raiders fans, unless Al Davis dies). In that vein, there's a block of people (say about 40ish%) that are going to pull the D lever pretty much no matter what, and a likewise percentage for the Republicans. That is, unless either side were to run a joke candidate (last time that really happened was Goldwater in 1964; it's unlikely today with the primary system that it would ever happen again - as much as I love Al Sharpton, he's never going to win the Democratic primary).
So what you are left with is maybe 15% of the population that is up for grabs each time. These people generally tend to be low-information voters (if they were high-information they would have already picked a favorite team, so to speak), and in surveys it has shown that low information voters use retrospective voting (i.e. am I better off today than I was four years ago?) conditioned on economic performance. Even if they give other reasons, often what has happened is that they blame or reward the party that has been in power during the last cycle and then rationalize it by citing other reasons (i.e. McCain's unserious, etc.)
Now, I suspect you're quite right that personality is not irrelevant; it probably accounts for some of the noise 1-2% either way (remember, that's a significant amount of the truly undecided) and in some elections that's enough to swing it. But given how bad the fundamentals were this year for Republicans, I think any of the top three Democratic contenders would have walked away with it.
That's not to take anything away from Obama and his team. He of course avoided the obvious trap of doing something stupid like get caught with a prostitute or whatever; they essentially ran a perfect campaign and had sure-handed and almost boringly on target operation. And he overcame fears about his race in a very skillful way; McCain and their campaign never found an angle against him that they could make stick. You can't really argue that there was much he could have done differently for the better.
-------
That's interesting, thanks. Yes as you said, I guess those LBJ and Nixon days of landslide victories (where the loser nets less than 100 electoral votes) are over. America is not as "united" as the candidates say, and 80% of us are firmly entrenched in our party affiliations. Despite all that Bush and Obama talk about "mandates" and "movements", the electoral map hasn't changed much since 2000, and the parts that changed usually swung less than 10% (which adds up to only a few million votes nationwide). And this change was apparently due to fundamentals, not "once in a generation" candidates or shifting ideologies. And as you said, it's kind of scary that national elections seem to be won or lost by the generally poorly-informed 15% of undecideds, who may just vote based on their pocketbook, last-minute feelings, or other visceral factors. I know those matters are legitimate voter concerns, but focusing on them might drown out other important information to consider (i.e. so many people voted for Bush just because they thought he would lower their taxes, and maybe others rejected Obama because they fear he will raise them).
I agree with you that public perceptions of giving full effort in this Puritan culture are important, or you end up like Fred Thompson. Maybe I'm just naive, but why is fundraising the measure of that effort to win? Shouldn't it be based on the merit of their proposals and rhetoric? Quality, not quantity. I think we'd all rather pick the quiet candidate who says smarter things and has a better track record versus the blowhard campaign-aholic with a better fundraising and marketing machine. And the success of that infrastructure is more attributed to the campaign team and political party, not necessarily the candidate's personal work ethic and cleverness.
On NPR last night, a funny vignette from a Montana voter: he said he was on the fence, but the Obama camp kept calling his home at dinnertime, so he voted for McCain instead. "Too much effort" can be bad as well. Remember that McCain beat out Romney, even though Mitt way outraised and outspent him in the early primaries. I think some voters do get suspicious, saturated, and turned off by "overzealous" candidates, and may favor a more pragmatic, tempered approach. Both McCain and Obama have accused each other of saying whatever it takes to get elected, and I guess that's nothing new. I know that campaigns by nature are all about excess and exaggeration. But there has to be some common sense and moderation too (values embodied by voters in states like NH and MN).
I agree that Obama ran a near-flawless campaign, but he had a lot of advantages to work with - especially Hillary messing up, money, volunteers, world events, and Bush. Not to say that he didn't earn it - he energized his supporters and reached more communities more than the GOP was able to. Biden was a solid pick and did his job (or at least didn't hurt him). His economic team was much better. I guess as you said, a "boring" campaign usually means a successful one without any gotchas. Message control, repetition, etc. It's strange that a "change" candidate also has to prove that he's really not that different. It's too bad that he had to go to such lengths to try to convince America that he is just like us, and played it so safe during the debates that we really didn't get a glimpse into his deeper political psyche. Apart from his great race speech (probably not even written by him, and only in response to the Wright scandal), he didn't really touch on ugly social injustice or other controversial, divisive matters like immigration. Of course he spoke of economic injustice, but that was an easy sell with rising health care costs, layoffs, and Wall Street acting like assholes. Team Obama was very innovative with fundraising, and harnessed technology much better than their rivals. Their Houdini software was able to identify the most receptive potential voters, like fishermen who use sonar I guess. They monopolized the change message that played out well for Americans frustrated from the Bush years, and didn't get too negative (well, front-runners don't really have to).
Though I don't know if some of their campaigning successes should be celebrated. From the BBC: "Everyone who visited the Obama website was asked to sign up to get more information. Everyone who did so was asked to contribute, or volunteer. If they did, they received several follow-up calls and messages asking for more money, or more assistance." Enough was never enough for them, and that's why they won. But is that what it takes to win these days? I know the GOP does it too, but it's out of control.
However, I will argue that the McCain could have done a lot better. Some commentators say that no Republican could have won in 2008. I know hindsight is 20/20, but of course he should have really evaluated Palin a lot more before picking her, to make sure she could handle the national stage and scrutiny of her personal life. I think Huckabee could have worked for him too, as more blue-collar and unconventional, yet a religious conservative with executive experience. McCain was badly hurt because he appeared (or is) out of touch with the economy and middle class needs. It's clear that both of them are not economics experts, but he didn't do a good job addressing that weakness. He came off as unstable and impulsive during the credit crisis, while Obama was always calm and reassuring. McCain was pro- and anti-regulation at the same time, though he did have good ideas for direct gov't restructuring of bad mortgages and not forcing retirees to withdraw from their 401(k)'s while stocks are down. Obama will probably adopt those measures.
McCain is old and Obama is hip; there's nothing the GOP can do about that, or the Obama fan worship. But at least he could have worked that difference to his advantage instead of just joking about it on Letterman. He can't "out-change" Obama, but he should have built on the steady experience argument that Hillary nearly won. He should have tried to make stick that Obama means well and gives uplifting speeches, but leadership takes more than that. Though more and more it appeared that McCain was actually the risky pick. During the debates, the tables were turned: it was McCain who looked childish, unpolished, and desperate with his attacks, while Obama was very collected and communicated clearly (well, he's just a better orator). McCain should have been the calm, sage Mondale lecturing the naive little boy Quayle on governance, but I guess that doesn't really jive with his personality. He has been around, he knows how Washington works and how to fix it, and Obama is over his head. I think he relied on his POW story too much when many undecideds don't care about distant Vietnam and feel detached from the current wars. He went overboard on offshore drilling (which in part cost him pro-environment voters at an 8-to-1 margin) and belligerence with Iran, especially considering his previous centrist record that independents used to like. He could have stressed his relationships with Lieberman, Feingold, and other Dems to get things done in Congress, which Obama couldn't match after just 3 years in Senate.
Handling Bush was a tough one. I still think that sitting presidents can be a major campaign help, even someone as unpopular as Bush. Gore rejected Clinton to his peril. Of course McCain could have politely demonstrated where he deviates from Bush (deficit spending, environment, Gitmo, stem cells, etc.). But he should have also celebrated where Bush and the GOP did well (no domestic attacks since 9/11, AIDS help to Africa, trade deals, conservative judges, more money for the military). He tried to bank on the Surge, but tunnel-vision voters only cared about the economy. Though he's not a Religious Right ideologue, maybe he could have done more on social issues where the GOP has success (or had Palin do it). Don't let the election boil down to one issue.
And finally there was race - it was such a slippery slope for the GOP to attack Obama for not being American enough, and created mixed-signals and problems like the crazy "Obama is an Arab" woman at his rally. I almost feel sorry for McCain to be thrust into such an awkward pickle. Surely he wants to capitalize on being more like the "guys on the dollar bills" than Obama, but also knows that he can't stoop too low (yet he has to rein in his supporters who do, Palin included). McCain has issues, but he's not a total jerk like Rove/Cheney, and never directly questioned Obama's American-ness. He tried to bring up Ayers, but not Wright. Why? Commedians and journalists alike have said that they were more cautious with Obama critiques, so as to not incur sensitive racial backlash, hence more negative media on McCain overall (well to be fair, he gave the media much more material than Obama). So in that sense, maybe racism (or the fear of being entangled in it) actually helped Obama. Also, McCain was the victim of personal lies in 2000, and I don't think he wanted to do the same and get so nasty about Wright, when that isn't really Obama's fault and doesn't reflect on his patriotism.
I agree with you about public financing (I'll abbreviate it as PF from now on to save keystrokes); it's a good way to restore more public control in campaigns. To explain his change of mind and refusal of PF, Obama said that public financing was "broken", and he needed larger sums of money to counter the GOP attack machine. But do he and the Dem Congress plan to fix PF now? I sure hope so. Actually I think PF is a bigger issue with Congressional races, where there are larger money disparities and huge incumbent notoriety advantages vs. presidential races. The system must be improved so it's at least decently competitive with private fundraising, or only the desperate would take it. I am not sure if you saw this link I forwarded a while ago, but what do you think?
http://www.clcblog.org/blog_item-57.html
