NewsLab Sidebar: How accurate are surveys in the Philippines?

(This is a sidebar to a main report, NewsLab Specials: Why the hate for pre-election surveys?)

MANILA, Philippines (NewsLab Team) - The past few weeks have seen Davao City Mayor Rodrigo Duterte surge in the pre-election survey race. The tough-talking Davao mayor pulled ahead of his rivals at 34 percent—a lead of 12 points that makes Sen. Grace Poe a far second.

With the election hitting fever pitch soon, many have begun to ask: Do the numbers dictate a Duterte presidency?

Experts were reassuring, "the race is still wide open," "it is a close fight," "anybody can win.” But if we are to look back—surveys in previous election seasons have gotten it right, more than once.

Independent research firms have repeatedly predicted the outcome of the national polls, according to a 2004 report by Philippine Center for Investigative Journalism’s (PCIJ) Yvonne Chua. She cited two instances—the 1992 and the 1998 presidential elections—where the Social Weather Stations (SWS) foretold the results. Gen. Fidel Ramos and Miriam Santiago neck-and-neck in the former (26.8 percent against 25); and Joseph Estrada via landslide (33 percent) in the latter.

We here at philstar.com tried to look at the numbers. Utilizing data obtained from SWS and Pulse Asia, we plotted how they were able to predict the outcome of two elections.

The first set of graph shows the presidential elections for the years 2004 and 2010, while the second tackles the vice presidential race.

The graphics above indicate that while Pulse Asia and SWS got the frontrunners right, they were barely close to the percentage of turnouts for some candidates. Most notable would be the overshoot in the Pulse numbers of Panfilo Lacson—by staggering gap—and Raul Roco.

Another noteworthy detail would be SWS and Pulse’s forecast of the leading two names: Gloria Macapagal-Arroyo and Fernando Poe Jr.’s shares were consistent with Comelec, showing them in a tight race. Although we also have to remember that this was the same election Arroyo figured in a wiretap controversy.

In 2010, Pulse Asia no longer exceeded the Comelec count in glaring fashion. The two research firms again accurately predicted the rankings.

Over at the 2004 veep race, the rankings turned out to be close to the Comelec’s as well. The single wrinkle would be Pulse and SWS both being unable to hit Loren Legarda’s numbers accurately.

Meanwhile, in 2010, Pulse again fell flat on anticipating Jejomar Binay’s solid lead. For their part, SWS was close to the Comelec turnout, and they, too, were able to get the order of rankings correctly—but only by a whisker. Pulse also had Mar Roxas as its front runner, along with overshooting returnee Legarda by almost 8 percent.

Scientific, systematic methods back the credibility of surveys—and this strength, in some cases, easily makes it a tool to sway public opinion.

De La Salle University’s Jaime S. Ong posited in a 2004 journal that research organizations or their field counterparts are corruptible. He went on to suggest that “as vulnerable as they are to sampling and non-sampling error, election surveys by reputable research firms provide as representative a snapshot, or series of snapshots, of voter sentiment as one can devise.”

Historically speaking, election surveys have been around even before dictator Ferdinand Marcos tweaked the Constitution. Opinion polling has been in the country almost the same time we gained our independence from the United States.

US firm Pew Research notes that while election polls attract a great deal of attention for their accuracy, their primary goal is to “help journalists and citizens make sense of the campaign and the election. “

In a Poynter Institute interview, FiveThirtyEight’s Nate Silver said that surveys “are used to provide a ‘veneer of science’ to media narratives.” The former New York Times data journalist added that “the basic problem is that people misinterpret random events as being meaningful."

A knee-jerk reaction on this analysis would be is why then would we still bother entertaining these surveys?

In Chua’s PCIJ piece, she noted how the general public still find it hard to believe that a small chunk of the voting population could paint the turnout of the actual elections. In fact, opinion polling has long met with resistance through the years.

For a recent example, Rep. Lito Atienza have called for the prohibition of publication of survey results. Two-time presidentiable and Sen. Miriam Defensor-Santiago have repeatedly slammed survey results, even labeling them as “mind-conditioning tools.”

So what do we make out of all these numbers? Retired National Statistical Coordination Board chief Dr. Romulo Virola has some bits of wisdom to share.

“In the literature on election polls, it is well-recognized that election polls can in fact influence voters in various ways,” he penned in an NSCB column. “responsible journalism can contribute a lot by publishing results only of surveys that pass standards of quality and professionalism.”

So those are the numbers geeks saying that the members of the media should refine these surveys into more than just a chunk of data.

And taking a bit from Mr. Ong’s parting shot: “[T]he outcome of the elections — flawed and bungled as they were — determines who gets to hold office for the next three or six years, regardless of what the surveys might have indicated. The rules of democracy require that this be so: it is the vote, not the survey, that determines the elected leader.”