There are surveys and there are those pretending to be surveys. It is election season and posting numbers indicating likely voting preferences generates social media clicks and traditional media attention.
People, however, get terribly confused. It is easy to fall victim to those who just want to create impressions for their clients.
I was required to study statistics and had some units in communications research in college. I also had to deal with market research findings while working in advertising and public relations. I have a good appreciation of what the numbers say and don’t say. But today, we will take it from an expert to explain what it is all about.
My expert is Dr. Jose Ramon “Toots” Albert who formerly headed the National Statistics Office. He is a professional statistician and a research fellow at the PIDS. He is just about one of the best resource persons on surveys.
Dr. Albert will deliver a lecture today at the Ateneo and he was gracious enough to let me take a peek so I can share his observations on the topic: “Survey Says or Not?” Almost all the views in this column today are his, and hopefully I do not misrepresent or misinterpret any.
Not all surveys are created equal. The surveys I trust are those that were scientifically designed and undertaken by trusted academics and professionals with reputations to protect. For me, it is Social Weather Stations and Pulse Asia.
But new survey outfits mushroom around election time. We have to be cautious about the validity of the numbers they present.
Dr. Albert said he was recently asked about a street-based survey claiming to have picked respondents randomly off the streets.
A good survey must have random sampling, but the random selection method must follow a scientific protocol. This random man-on-the street survey is not random enough for a survey to be trusted and to be useful.
“Going out on the street and randomly picking people will not work to get a national reading or even a reliable reading of those areas because not every voter in a city may pass through the street,” Dr. Albert explained.
Randomness in the scientific sense means every potential respondent in a universe or population being surveyed must have an equal chance of being selected.
This is also why Google trends are also unreliable for tracking national sentiment because not everyone is on the net and not everyone on the net knows how to Google. At best, Dr. Albert says, Google trends will reflect the AB crowd. He also warns that such data is likely to be manipulated by trolls working for some candidates.
To help us evaluate the value of surveys, Dr. Albert quotes from a 1953 book on How to Lie with Statistics. Chapter 10 of that book suggests asking a number of questions:
First: Who says so? (Who conducted the survey? Who paid for it?)
Second: How does he know? (How was the survey conducted?)
Third: What’s missing? (How many respondents were interviewed? What was the margin for error? How were questions asked?)
Fourth: Did someone change the subject? (Are descriptions of data and conclusions based on the data?)
And last: Does it make sense? (What assumptions are made in the subject report and analysis?)
Survey data, Dr. Albert explains, do not tell us directly why people approve or are satisfied with a leader’s performance. Some pundits may try to explain that situation to “fear factor.”
In a recent webinar, SWS fellow Geoffrey Ducanes expressed curiosity about how much of satisfaction may be influenced by personal experience, misinformation, and propaganda. “How much is driven by fear of expressing dissatisfaction so what is observed is not real satisfaction.”
Pulse Asia’s Ronnie Holmes and several fellows of SWS admit that fear cannot be ruled out. But measuring fear can be challenging.
Dr. Ana Tabunda, senior research fellow of Pulse Asia, and Steven Rood of SWS, according to Dr. Albert, said that there are currently no nonverbal signals from their survey respondents to support the “fear factor” conjecture.
Still, Dr. Albert pointed out, half (51 percent) of Filipinos in the SWS July 3 to 6, 2021 National Mobile Phone Survey agree that “It is dangerous to print or broadcast anything critical of the administration, even if it is the truth.” But the danger may just be for media practitioners and the survey respondents may feel they can safely say what they want to a survey interviewer.
Dr. Holmes and some Japanese colleagues recently delivered a paper based on an experiment they undertook that looks into the issue.
They found empirical evidence that about one-third of Duterte’s approval rating came from social desirability bias (i.e., respondents are not giving truthful answers in surveys). Filipinos who believe their neighbors support Duterte were more likely to succumb to this social desirability bias.
According to Dr. Albert, “the implication of this fascinating paper would suggest that support for Duterte may wane quickly when people perceive he is no longer popular.”
Dr. Albert also said that “a high support for authoritarian populist leaders should not be interpreted as an indication that voters are giving up on democracy, as some pundits have argued.”
Dr. Albert said that SWS’s Ducanes suggests from an attempt to model explanations for the approval rating, that people who saw Duterte as “decisive”, “diligent”, and “authentic”, specially the first two traits, were more likely to be satisfied with him.
“It is, thus, the character of the President that makes him very popular: Du30 is the familiar kanto boy, Digong Tigas, who is bastos, but authentic, and most of all entertaining, and we Filipinos love entertainment.”
Dr. Albert reports that “the model of Ducanes that uses perception of character traits of the President was reported to correctly predict 99 percent of those who were satisfied and 45 percent of dissatisfied respondents.
A Pulse Asia focused group discussion (FGD) also revealed a perception Duterte is hardworking for making speeches late at night.
More on surveys in a future column. Suffice it to say, be careful of so-called surveys that were designed for bandwagon effect to win over the unwary voters.
Boo Chanco’s email address is bchanco@gmail.com. Follow him on Twitter @boochanco