There will always be an audience for a particular type of story about the 2016 election — claims that Russia’s social media activity powered President Trump’s eventual victory. There is a sizable portion of the American public that believes Trump’s electoral-vote victory was illegitimate; some large percentage of that group will readily attribute the win directly to Russia’s efforts.
We should note at the outset that it’s clear that Russia’s interference in the election had a tangible effect. The information stolen from the Democratic National Committee and Hillary Clinton’s campaign chairman that was later released by WikiLeaks was a staple of media coverage around the conventions in July 2016 and during the last month of the campaign. While measuring the effect of that leaked information is tricky, it’s clear that it had influence.
That social media push, though? Well, it was more compelling as a narrative, certainly, involving murky, modern terms such as “bots” and “targeting.” But, as I’ve written before, there’s very little evidence that Russia effectively targeted American voters with messages that powered Trump’s victory. Russia paid for a lot of Facebook ads in the populous states of New York and Texas in the last five weeks of the campaign, but its ads targeting the three states that handed Trump the election — Michigan, Pennsylvania and Wisconsin — were seen by only 1,000 people. There’s no evidence at all that Russia used Twitter to target people in particular places or demographic groups, targeting that would have left fingerprints in the form of receipts for payment.
Maybe the Russians used algorithms or advanced propaganda techniques to flood the conversation by indirectly targeting people and amplifying messages with bots! It’s hard to debunk this theory because doing so requires first understanding what automated tools can and can’t do and how voters are influenced in campaigns. It requires less work to accept that nebulous assertion about Russian deviants deploying nefarious code than it does to explain why it doesn’t make much sense.
But, again: It’s a compelling idea! And, lo, we arrive at a new study, reported by NBC News and Axios, that suggests a link between Russian social media efforts (specifically on Twitter) and Trump’s support in the polls during the election campaign. While both news outlets drew the standard and important distinction between correlation and causation — that is, that just because things happen in tandem doesn’t mean that one caused the other — both also hype the study as showing exactly the link implied above.
“New study shows Russian propaganda may really have helped Trump,” NBC News’s headline reads. “Study suggests Russian social media trolls had impact on 2016 election,” Axios agrees.
So what does the study say? Well, the research team from the University of Tennessee took tweets identified by Twitter as coming from accounts linked to the Russian Internet Research Agency and matched them to Trump’s poll numbers, as tracked in polls compiled by FiveThirtyEight. The central finding is that “a gain of 25,000 re-tweets per week over all IRA tweets (or about 10 extra re-tweets per tweet per week), predicted approximately one percent increase in Donald Trump’s poll numbers.”
That’s evocative, certainly. The report includes a variety of data points and a number of graphs like those below. They show the number of tweets from IRA accounts (A), poll support for Clinton and Trump (B), a comparison of the average number of retweets with Trump’s poll numbers (C), and a comparison of the average number of retweets with Trump’s position in the polls (D).
This last graph is particularly important. That green line shows the trend: As the average number of retweets climbs, so does Trump’s position in the polls.
So let’s look at it more closely. Here, we’ve used data shared with the digital media outlet the Conversation to code results by time period and to identify some particular weeks. (Circles are also larger as the week at issue gets closer to Election Day.)
“Compared to its time-average of about 38 percent,” the report says about this graph, “support for Trump increased to around 44 percent when IRA tweets were at their most successful.” That’s a reference to the July 14 data point shown above: When the IRA tweets had the highest average retweet number, Trump’s position in the polls was at its highest.
But it was also similarly high in late November and early December 2015, when the IRA tweets averaged very few retweets. In fact, the correlation between the two sets of data isn’t really that robust. It has an r-squared value of 0.27, according to the data used for the above graph. The closer an r-squared value is to 1, the more robust the correlation. In other words, the correlation here isn’t that strong.
You may have noticed in Graph C above that there was a big spike in retweets at the same time that Trump saw a spike in the polls. That’s the July 14 week. It’s not clear exactly where weeks start and stop in this analysis (July 14 is a Thursday). But it is worth noting that Trump’s spike in polling in late July was a function of the Republican convention, which ran from July 18 to July 21.
That said, the spike on the graph above doesn’t actually correspond to the spike shown in FiveThirtyEight’s data. The RealClearPolling average, like FiveThirtyEight’s, had Trump peaking shortly after the convention, not right before. In FiveThirtyEight’s poll average, Trump never had more than 38.3 percent support from July 7 to July 21. So where does this 44 percent poll value come from?
The researchers appear to have averaged all of the polls compiled by the site. That means that the second-most-represented poll in its data came from USC Dornsife and the Los Angeles Times — a poll that was one of the least accurate in 2016 thanks to its consistently overstating Trump’s support.
Trump’s support in the researchers’ data rose more than three points from the week of July 7 to the week of July 14. It’s not clear why.
It’s important to note that the researchers focused on retweets and not overall tweets from the IRA. (In fact, they found that “we see weak evidence for an effect in the opposite direction, suggesting the possibility that IRA Twitter activity is increasing in response to Trump’s polling.”) This suggests that, if there was a meaningful correlation between Twitter activity and poll data, both were driven by some outside engagement. People becoming active on Twitter also may have happened as they were demonstrating more support for Trump. This is what’s known as a causal fork: Both the IRA retweets and Trump support may have been caused by the same external thing.
If there’s a correlation here, that is. Which is . . . up for debate.
It’s important to note that, on its face, the idea that 25,000 retweets could drive national political polls by a percentage point seems highly unlikely. Over the course of the 2016 election, there were 75 million tweets directly related to the election itself. If only 1 percent of those were retweeted 10 times, that means that the 25,000 retweets are fitting into a flood of 75 million original and 7.5 million retweeted tweets. It means, in other words, that the requisite 25,000 retweets make up 0.03 percent of all of that Twitter activity.
(To give a sense of scale: A study by Symantec found fewer than 120,000 tweets and retweets of IRA tweets in October 2016.)
That’s only for election-related tweets, mind you. Most of the IRA tweets (like most of their Facebook ads) were qualitatively something else. For example, the tweet that received the most retweets among those offered by the IRA before the election was one from a troll posing as a liberal black woman showing a video of a woman accompanied by text, noting “the pain in her voice.” It’s not clear what the video was (Twitter removed all of the trolls’ content), but it appears to have been part of the effort to stoke division by highlighting issues of race.
If the 97,498 retweets it received all came in the same week, would this have spurred a three-plus-point rise in Trump’s polls? Or would other most-retweeted tweets, several of which similarly focused on issues of painful black experiences in the United States?
We also have to note that these were not targeted tweets, in any meaningful sense. Several of the accounts with the most tweets were named after geographic locations in an apparent attempt to appear to be legitimate news outlets in those places. The locations? Chicago, Newark, San Francisco and Kansas. None of these were really up for grabs in 2016. Chicago and Newark were probably chosen because of their large black populations, reinforcing that the goal was often as much racial division as Trump’s election.
It will be interesting to see whether other researchers are able to replicate the analysis undertaken here. We certainly can’t definitively say that no votes were changed as a result of Russian disinformation on Twitter or that no one’s political views were influenced by it. We can say, though, that this study is worth a great deal of skepticism — especially among those who are looking for evidence that Russia’s trolling handed the election to Trump.
