Saturday, November 20, 2010

And still counting...

Predicting the future

The 2010 general election is not yet over in California. The secretary of state's office in Sacramento continues to issue updates as it aggregates the returns trickling in from California's fifty-eight counties. As of the last report, time-stamped 5:00 p.m. on Friday, November 19, twenty-six counties still had untallied ballots to count. By adding up the counties' estimates of vote-by-mail and provisional ballots, the secretary of state announced that approximately 629,634 votes remained to be processed.

It's not a moot point. While attorney general Jerry Brown trounced Meg Whitman in winning his third term as governor and Barbara Boxer convincingly defeated Carly Fiorina on her way back to the U.S. senate, one statewide race remains too close to call. The Democratic nominee to succeed Brown as attorney general currently has 4,291,854 votes to her Republican rival's 4,248,804, a margin of only 43,050. Taking into account a scattering of votes among minor party candidates, that breaks down to 46.0% to 45.5%.

Democrat Kamala Harris and Republican Steve Cooley have swapped the lead back and forth a few times since the vote tallies began to be published after the November 2 election. Cooley actually declared victory election night (see the video below), but woke up the next morning to discover that Harris had edged ahead. When the vote count moved him back into the lead a few days later, he was smart enough not to make yet another premature victory speech. When Harris regained the lead, she prudently kept her own counsel.

My own opinion? For a couple of weeks now, I have been expecting a Harris victory. It's in the numbers.

I looked into the numbers because one of my friends, a retired journalist, was scoffing at the superficiality of the news articles on the election results in the attorney general's race. Except for striving heroically for different ways of saying “too close to call,” none of them offered any substantive analysis.

“It all depends of where the remaining votes are,” he said. “Instead of just paraphrasing the press releases from the candidates, the reporters ought to dig into the details. They should do some reporting.”

He prodded me into action. I downloaded the secretary of state's report on unprocessed ballots (well over two million at that time) and loaded it into a spreadsheet. Then I perused the secretary of state's report on the percentages accruing to each candidate in each county. By way of example, consider Tulare county, where California's most conservative voters gave Harris only 29.8% to Cooley's 62.4%. Tulare's county clerk estimated that 3,350 ballots remained to be processed. Applying the percentages to this number, I computed that Harris would get 998 more votes and Cooley would get 2,090. (I'm sure Mom & Dad's vote-by-mail ballots are in the latter batch.)

I applied this process to all of the counties with outstanding ballots, obtaining an estimate for the additional votes likely to be obtained by Harris and Cooley. Upon adding the estimates to the votes counted to date, I found myself looking at a razor-thin Harris victory. Every so often I would return to the secretary of state's website to tweak the percentages to reflect the completed count. Those numbers were very stable, seldom moving more than one-tenth of a percent. The predicted Harris margin varied, but never vanished.

My latest computation, based on yesterday's numbers, suggests that Kamala Harris will defeat Steve Cooley for the office of attorney general by 45,902 votes. I'm not sure about the 2, though.

If the numbers hold up, the Golden State will have handed the Democratic Party a clean sweep of every statewide office. May it make the most of its opportunity.

Note: I should give a tip of the hat to Timm Herdt of the Ventura County Star. He had the same idea that I did and published his estimate on November 9 on his blog. In my opinion, however, Herdt pulled up just a bit short by confining his attention to the 21 counties with the most votes remaining to be processed. In so close a contest, it was unwise to scorn the little counties and risk that much round-off error. On the basis of his computations, Herdt figured that Cooley had an edge.

While I obviously think Herdt was wrong, my ex-journalist friend can be relieved to learn that at least one reporter is willing to go digging for news. It's not quite obsolete yet.


Kathie said...

Zeno, it might make an interesting topic for a scholarly paper to apply your methodology to past close elections (e.g., Franken vs. Coleman, MN., 2008) to see how well it holds up around the country. While I suspect you're right, the only way to tell would be to disprove the null hypothesis.

I also wonder if you could do a study on the Alaska Senatorial write-in ballots this time (although the only comparator is Strom Thurmond in SC in 1954, and I have no memory of it, nor any knowledge of how much documentation survives).

Kathie said...

Another correlative factor might well be whether a state's vote-by-mail option is widely-used (like in California and a few other states).

Whereas, Pennsylvania does everything in its power to dissuade voters from casting ballots prior to election day, and grants absentee ballots only a week or so in advance -- and only to those who will be away that day or who have medical excuses indicating that going to the polls would pose a significant burden.

I suspect that where early voting-by-mail is widely practiced, the voting profile rather closely follows that of election-day voters (as you posit in your post). However, here in PA. where it's less commonly done, absentee ballots tend to skew more conservative than the electorate at large, since those likeliest to use the option are often either business people or the elderly.

I imagine, though, that you can factor such considerations into your prediction methodology for states where applicable.

Zeno said...

Years ago, when absentee voting was less common in California, a Democrat could normally breathe a sigh of relief if he carried the early-count absentee votes. Those votes were almost always less generous to Democratic candidates because they skewed toward affluent Republicans. Now, however, vote-by-mail ballots more closely mirror the general electorate. Many more people are using them and some voters are actually required to. (Many rural precincts have too few voters to justify the expense of a polling place. Those voters automatically receive mail-in ballots at election time.)

In addition to simple popularity, vote-by-mail is also skewed by the efforts of some campaigns to get out the vote early and "bank" votes for their candidates. A good operation, like the one Sen. Reid had in Nevada, can blunt the effects of supposed voter enthusiasm—which everyone attributed to Sharron Angle's supporters and not to Reid's—and make the difference in a tight race.

Kathie said...

Zeno, have you ever seen any research or polling on whether early balloters wind up feeling "voter remorse" because by election day they wish they'd voted for a different candidate?

Or do early voters tend to be the most confirmed in their choice, such that little if anything could sway them to change their minds?

Karen said...

I'd also be interested in knowing whether election-day dropoffs of mail-in ballots skew one way or another. I've been doing "mail-in" ballots in California for a couple of years now, and I can't ever get the damn things mailed in time so I drop them off at my polling place.

Billy C said...

You said:
"(Many rural precincts have too few voters to justify the expense of a polling place. Those voters automatically receive mail-in ballots at election time.)"

Do we know how this has affected participation?

The Ridger, FCD said...

After 18-20 months of a campaign, i would think you'd need something massively scandalous or or life-altering to really change your mind - at least on the big races (unless you cast a protest vote of some sort thinking your candidate was a lock and found out the race was closer than you'd thought).

What could you learn in the last week?

Kathie said...

Ridger, the "October surprise" is a time-honored tradition in general elections, some damning piece of info that's revealed only a week or so before election day, so the target hasn't time enough to perform sufficient damage-control.

Zeno said...

No, Billy C, I don't know how vote-by-mail has affected participation in those precincts where it's mandatory. The secretary of state's office probably has data on it, but I don't recall having seen it. Washington State now does a lot of vote-by-mail and might be an even better source of information on its impact. I recall that the Democratic incumbent actually carried the absentee vote, so it's clear there was a sudden last-minute swing against him.

Ridger, I can report one case in which a district attorney in cahoots with the GOP indicted a California state senator the week before a similarly named senator was up for re-election. Many observers (and I'm one of them) think the confusion over the names and the indictment cost the senator the election, leading to the installation of a sleazy young right-winger as his successor.

The Ridger, FCD said...

I ask about the "remorse" question because, although I always hear about "the October surprise", I can't remember a single actual example. Surely this doesn't mean they don't exist - in fact, I was very apolitical for a very long time after the RFK assassination, and plenty of them could have been set off - but nowadays the campaigning starts so early, and people are counted out so damned early (two primaries in!), and coverage is so intense, that it seems to me, on no hard basis, unlikely for a major race to be derailed like that.

Kathie said...

AP: Jerry McNerney holds on to win third term:

"Rep. Jerry McNerney, D-Pleasanton, has been re-elected to a third term, fending off a challenge from Republican David Harmer. ¶ McNerney held a lead of nearly 2,500 votes on Wednesday with less than 1,900 ballots left to be counted. ¶ His re-election means no California congressional seat changed party hands..."