Repeat Offenders: Voting Misinformation on Twitter in the 2020 United States Election
Authored by the Election Integrity Partnership Team
Key Takeaways
Several domestic verified Twitter accounts consistently amplify misinformation about the integrity of the upcoming election.
These ‘repeat offenders’ often source misinformation from hyper-partisan media outlets, including the Gateway Pundit and Breitbart News.
These hyper-partisan media outlets and “repeat offenders” often exploit local news outlets who report factual news by reframing that news to be misleading or biased.
Platforms should consider enacting stronger sanctions so that repeat offenders are more obvious to the public — e.g. labels signaling that certain accounts repeatedly share misleading information about voting — to help curb the spread of misinformation on their platforms.
Introduction
On Sept. 8, the Election Integrity Partnership (EIP) began its efforts to detect and mitigate the impact of attempts to deter people from voting or to delegitimize election results. This work involves reviewing hundreds of reports of potentially misleading voting information being spread on social media. These reports were submitted by election officials, civil society partners such as the AARP and NAACP, and EIP analysts tasked with monitoring social media. A significant percentage of these reports have featured misleading narratives about mail-in ballots and voter fraud, claims that have been tied by other researchers to a coordinated disinformation campaign.
In this blog post, we want to highlight patterns we have noticed across these various investigations over the past six-weeks. We look back at the set of misleading stories the EIP reviewed to identify which Twitter accounts consistently share false or misleading content about the upcoming election, and from where these accounts source this misleading information. We scoped our investigation to misleading stories surrounding voting. For example, EIP analysts found that certain real events, like those involving mail-in ballots or the USPS, are often recontextualized to support false narratives delegitimizing the election. There is evidence that narratives such as these have effects on voters’ behavior. In general, these false narratives suggest that widespread voter fraud will undermine the results of the 2020 General Election; however, experts agree that there is no evidence to support these claims.
Among the hundreds of misinformation reports investigated by the EIP, we chose to focus on cases that met specific criteria: 1) entered the EIP systems between Sept. 8 to Oct. 20; 2) related to voting concerns — e.g. mail-in voting and/or voter fraud; 3) were labeled by our team as high priority based on metrics of severity (i.e. negative impact of the false/misleading information) and propagation; and 4) had significant spread on Twitter. 43 cases met these criteria.
As part of our investigations into each “case” of election-related misinformation or disinformation (categorized as procedural interference, participation interference, or fraud), EIP analysts identify links to related content shared online. In this analysis, we used these links, along with combinations of terms associated with each story, to identify tweets related to these stories. We identified these tweets from within an existing database — an ongoing, real-time collection of voting-related tweets maintained by the Center for an Informed Public at the University of Washington. The voting-related tweet data is collected by sampling public tweets containing terms such as “vote”, “ballot” and “mail-in”. For example, to find tweets by users referencing an event where 400 Michigan military voters received ballots with the incorrect running mate, we searched for tweets within our voting-related dataset, scoping to tweets that also contain the terms “military,” “misprint,” or “michigan” from within a specified time range after the story first was published. The dataset used in the remainder of this blog is therefore all tweets from our ongoing voting-related collection related to (as defined by shared links or keywords) the 43 selected cases featuring misleading narratives about voting, previously investigated by the EIP.
Though a few of these cases included propagation of misleading narratives on the political left, especially in connection with criticism of the US Postal Service, the vast majority of the selected cases featured stories that were amplified on the political right — feeding existing meta-narratives tying mail-in ballots to systematic voter fraud.
In our analysis of this collection of tweets mentioning the 43 cases, we found that a relatively small number of verified, domestic Twitter accounts are responsible for repeatedly sharing voting misinformation to a large audience this election season. We found that two media outlets are especially large sources for misleading information about mail-in voting and voter fraud, and that the reporting from local outlets is repeatedly re-framed (by others) in misleading ways despite accurate reporting initially. We found that a surprisingly small percentage of users account for most of the misleading information about these stories shared on Twitter. For this article, we are calling the accounts that consistently shared voting misinformation “repeat offenders.”
Repeat Offenders with Large Audiences
We begin by investigating to what extent “repeat offenders,” or accounts which are repeatedly involved in growing these narratives, are influential in helping to promulgate the spread of these misleading narratives. In other words, across the 43 misleading narratives about voting investigated by the EIP, are there Twitter accounts that repeatedly share and engage with these misleading narratives. We distinguish, in our analysis, between original content (tweets authored by the posting account) and retweets (posts that are re-shared by others). Across our dataset relating to these 43 incidents, more than 50% of all retweets can be traced back to “original” tweets from only 35 users (out of over 600K users in our dataset). Some of these 35 users only had one highly retweeted tweet, but most had multiple tweets across different incidents sharing misleading stories.
Some users not only produced a lot of retweets, but were connected to multiple distinct misinformation narratives. To isolate users who repeatedly engage with misleading information to a wide audience, we extracted all accounts that had posted at least three original tweets containing information about and/or references to any of the 43 misleading election stories, and for whom each of those tweets got at least 1,000 retweets. The 20 users who meet this criteria are shown in the table below:
Table 1: Twitter accounts that repeatedly shared misleading information about mail-in voting and voter fraud. Non-verified users, with the exception of John Solomon (manager of Just The News), are anonymized.
Out of these 20 accounts, three are associated with partisan news sources on the political right, which have been linked in the past to spreading misinformation: Breitbart News, Jim Hoft (founder of the Gateway Pundit), and John Solomon (founder of Just the News). One represents Breaking911, a breaking news aggregation account that frequently traffics in misinformation. The remaining nine accounts include Charlie Kirk and Benny Johnson, who are associated with the conservative organization Turning Point USA; Tom Fitton, who is the founder of conservative activist group Judicial Watch; the personal account of current U.S. President Donald Trump; and several right-wing Twitter influencers including James Woods and Josh Caplan.
Taken together, posts from these 20 users are the source (original tweet) for approximately 20% of all of the retweets in our dataset. This means that a small number of accounts is responsible for a large portion of the spread of misleading election-related information. In the section below, we describe some of the tactics these accounts use to shape and share misleading information. Specifically, these accounts reframe existing stories, decontextualize trending stories, exploit local news coverage, and, in the case of media outlets, produce their own misleading content.
Reframing
Every repeat offender identified in this analysis engaged in reframing. By adding misleading or false statements to what was originally a true story, these accounts were often able to craft their own narratives, using media stories and factual accounts of incidents as the raw material for these misleading messages. For one particularly illustrative example, consider a quote tweet referencing a video which, according to reporting by the Philadelphia Inquirer, depicted “residual print production waste in a trailer that was returned from a recycling facility.” This tweet was authored by James Woods, American actor and producer who has recently become known for expounding his right-wing political views on Twitter. In direct contrast to the facts as originally reported, accounts on Twitter had been circulating this video claiming that it depicted “shredded mail-in ballot applications” filled out for Donald Trump. In his quote tweet, Woods wrote “Ballot fraud on video…” and received over 2,000 retweets (the tweet was later removed).
Figure 1. Screenshots showing how Twitter users created false narratives of voter fraud from a video depicting waste from a recycling facility
By following the chain of attribution ending with Woods’ quote tweet, we can see how different Twitter accounts collaborated to reframe a story, wittingly or otherwise. The tweet that Woods quoted was authored by a non-verified user who had 40,000 followers and has participated in (either tweeted or retweeted content) 21 of the 43 misleading voting incidents studied here. This user (anonymized with dark orange in the above image) was, in their tweet, sharing a video tweeted by another non-verified user (green) who had over 10,000 followers and who shared five different misleading voting incidents on Twitter. Here they posted a video with a screenshot of a tweet by a third non-verified user with over 40,000 followers (gold) and no other observed history of sharing misleading voting misinformation.
With each duplication, the next user added new and misleading framing to the issue at hand, distorting it from its original context. Green screenshotted the original tweet from gold, making it impossible for future users to know that the original tweet has in fact since been deleted (presumably for misleading content). Dark Orange added baselessly that “thousands” of ballots were shredded, and claimed that the video is “evidence,” implying that a crime had taken place. Finally, James Woods stated directly that the video is “ballot fraud on video.” Through their collective reframing, they turned a confusing video of unknown origin into seemingly concrete but ultimately false evidence of ballot fraud in Pennsylvania.
Decontextualization
In addition to reframing, many users rip information from its original context, making claims more difficult to fact-check and interpret in light of other evidence. This is decontextualization. We saw this in the previous example, with the user who screenshotted an original tweet so that subsequent users would not know it was deleted. Charlie Kirk, Chuck Callesto, and Josh Caplan consistently decontextualize in their tweets, making it easier for false or misleading content to spread without correction.
For example, consider the widely-spread but unsubstantiated claim that Rep. Ilhan Omar (D-MN) was associated with an illegal scheme to exchange cash for ballots ahead of the presidential election. EIP researchers investigated this story and believe it to be part of an ongoing disinformation campaign to create a false impression of widespread voter fraud. A local media outlet reported that the provocative video used to launch the initial claim was edited to remove important context and that some participants in the video had been offered bribes for their appearances. To our knowledge, there is no evidence tying Rep. Ilhan Omar to any of the alleged wrongdoing.
Figure 2: Tweets repeating misleading claims in an apparently coordinated manner
Nonetheless, Charlie Kirk, Chuck Callesto, and Josh Caplan each crafted extremely popular tweets — with no external links or sources — referencing this purported ballot harvesting story. Charlie Kirk asked his audience, “Ilhan Omar should be expelled from Congress and immediately investigated for her role in the Minnesota ballot harvesting scheme. RT if you agree,” and received over 46,000 retweets. Chuck Callesto tweeted “REPORT: Ilhan Omar looks to be connected to a CASH FOR BALLOTS harvesting scheme… MEDIA SILENT..” and received over 8,000 retweets, while Josh Caplan tweeted a follow-up story “FOX NEWS: Minneapolis police investigating alleged ballot harvesting scheme linked to associates of Rep. Ilhan Omar,” both without any external links.
By excluding external links from their tweets, these users make their content more difficult to put in context, and thus more difficult to fact-check. If Caplan had linked to the Fox News story he referenced, his followers would have read that Project Veritas (the producer of the initial view) was “a controversial group that has produced a number of misleading videos,” and may have treated the news with more skepticism. Kirk and Callesto both allege that Rep. Omar is “connected” or had a “role” in the story reported by Project Veritas, something that no mainstream outlet has ever reported (to our knowledge). By depriving their followers of explicit sources with which to verify the information they are sharing, they encourage the spread of what appears to be a disinformation campaign aimed at undermining faith in the upcoming election.
Rewriting Stories: The Role of Partisan Outlets
Three of the repeat offender accounts we’ve identified represent right-wing, partisan media outlets (Breitbart, The Gateway Pundit, and Just the News), and one account, Josh Caplan, is an editor for Breitbart. These accounts often go further than resharing or decontextualizing content: they also rewrite news stories originally reported by other outlets, and republish them with a new and often misleading partisan framing. In the stories we’ve analyzed, these outlets modify headlines and add sentences to suggest that individual election and mail irregularities represent widespread fraud and electoral uncertainty. These outlets were outliers from the other news organizations in our dataset, because they both appeared in many different incidents that we analyzed, and their content was often shared widely on Twitter.
Prime among these partisan outlets were the Gateway Pundit and Breitbart. These sites were cited in 21 and 19 of the 43 incidents, respectively, making them the first and second most-referenced website domains in our dataset. Across all incidents, Breitbart was retweeted over 151,000 times and the Gateway Pundit was retweeted over 117,000 times, again first and second among all the websites we analyzed. They had multiple stories shared widely across Twitter, with 10 stories from the Gateway Pundit reaching over 1,000 total retweets, and six stories from Breitbart reaching the same threshold, again first and second in our dataset. Only 12 other media outlets had more than one such story, 8 of which were also partisan sites doing little to no original reporting (Just the News, The Sara Carter Show, The National File, the Epoch Times, The Federalist, Hannity.com, Jeffrey Lord, and the Washington Examiner).
Figure 3: Collection of headlines from Gateway Pundit articles that contained misleading information and framing
For an example of Breitbart’s participation, we can return to the Detroit News’ story on “400-plus Michigan overseas ballots list wrong running mate for Trump.” The Detroit News published this story on Sept. 15, during which time it got relatively little engagement on Twitter. One day later, on Sept. 16, Breitbart wrote their own version citing the Detroit News, “Democrat Michigan Secretary of State Misprints Trump Ticket on Ballots for Troops.”
Figure 4: A plot of the number of retweets linked to misinformation about Michigan overseas ballots. Selected tweets with large numbers of retweets are highlighted, with arrows indicating the time of the tweet. Two accounts are anonymized because the users are not verified.
The original Detroit News article garnered little attention on Twitter. Some users attempted to reframe the article to suggest deliberate fraud by the Michigan government in misprinting these ballots, but those tweets did not receive much engagement. It was only when Breitbart published their own version did the story spread on social media. Twitter users like Dan Scavino and other non-verified Republican and/or pro-Trump Twitter influencers shared the Breitbart link, and were retweeted thousands of times before Breitbart’s own Twitter account shared the story, leading to even more attention. As the story spread, others began to suggest that people in Michigan state’s government had intentionally misprinted these ballots (which was expressly denied by the state’s government), and Charlie Kirk gave the story another boost using this framing. Finally, President Donald Trump retweeted Breitbart’s link from his personal account, giving the story much more exposure than it had previously achieved and baselessly claimed that the ballots were misprinted “illegally and on purpose.” By the end of this process, this Breitbart article was the fourth-most shared URL in our misleading stories dataset.
The Breitbart article in this example served two purposes. First, in its text, it reframed the story to focus on the actions of the “Democrat” Michigan government, and suggested those actions might be due to bias, writing: “the secretary of state has endorsed Biden for president.” We have previously noted that a common technique used to spread misleading narratives about voter fraud is to falsely assign intent to otherwise non-political actions or mistakes. Second, Breitbart redirected the story to an audience of right-wing influencers who were eager to reframe the story as a willful attempt by Democrats to subvert the electoral process. It was spread first among pro-Trump and/or right-wing Twitter influencers, and then by President Donald Trump himself.
Exploiting Local News Outlets
The final tactic we observed was the exploitation of local news. Many of the most widely-shared misleading tweets in our dataset linked to true stories reported by local news outlets. These outlets reported on isolated stories relevant to their local audience, such as irregularities in local elections or incidents of improper mail disposal. Some examples in our data include Philadelphia Inquirer’s article on a statewide extension for mail-in ballot return in Pennsylvania and the Atlanta Journal Constitution’s article on 1,000 voters who had voted twice in their summer primaries (they later clarified in a follow-up article that these voters may have voted twice, but there was “no conspiracy”). The stories they wrote were often true to the extent that information was available at the time of reporting, and did not draw conclusions about voting during the election nor link to larger narratives of election fraud. Once their stories were posted, however, politically-motivated social media users reframed them to focus on the presidential election at large, often extrapolating from the stories to claim things that were demonstrably false. Five out of the top 25 most-shared websites across the incidents we analyzed were from media outlets with a local focus.
Take, for example, this story by KTLA, a local news outlet based in Los Angeles and serving Southern California. KTLA reported on a surveillance video that appeared to capture a person disposing of bags of mail in a parking lot in Glendale, California. What follows is a typical pattern that we’ve observed for election stories by local media outlets; Twitter accounts that have repeatedly shared misinformation decontextualize and reframe the story into something deeply misleading.
Figure 5: A plot of the number of retweets a story by KTLA on a surveillance video of mail being dumped in a Glendale Parking lot. Selected tweets with large numbers of retweets are highlighted, with arrows indicating the time of the tweet. The first account is anonymized because the user is not verified.
In the first two days since it was published, this story had gained little traction on Twitter, before beginning to spread, initially on the political left, in connection with ongoing criticisms of the post office and claims that President Trump was actively trying to diminish their capacity to deliver mail-in votes. Catalyzing this initial spread, an unverified, left-leaning user shared the link with the following framing: “It’s crazy that we can’t even trust that our votes will be counted if we use the USPS. I would rather risk getting #COVID & vote in person!” As displayed in the graph above, this first tweet used a deeply misleading framing — that mail-in voting is ineffective — to spread KTLA’s story into a large community that may otherwise never have read its coverage. A day later, verified user Don Winslow (@donwinslow) shared the link twice, once with the headline copied and a request to “please watch and share this,” and another six hours later with the framing “Please Dont Trust [USPS Postmaster] Dejoy With Your Vote.” These tweets seemed to spark a new streak of retweets for the next 24 hours with the same misleading framing.
After that, the story seemed to die down for a number of hours, only to be resurrected, this time on the political right, by verified user Tom Fitton (@tomfitton), who shared the link with the caption “Harder to trash your ballots if you vote in person.” Notably, this user, who is on our top “repeat offenders” list, shared the story to a predominantly right-leaning audience, as opposed to the likely left-leaning audience of the previous two users. This exposed the misleading narrative to a whole new cohort of users a full five days after it had originally been published. From barely being retweeted during the first two days after publishing, these reframings likely contributed to this KTLA story to being in the top 30 most shared URLs in our entire dataset.
Despite the different political leanings of these accounts, their posts both served to reframe a local news story as indicative of widespread failures in the mail-in voting system, and encourage people to vote in person. Given the potential risks due to COVID-19 that some people face with in-person voting, preserving faith in mail-in voting as an option is essential to maintaining access to voting for all. This misleading narrative piggybacked on a well-reported local news story to achieve a viral spread that would otherwise be unlikely.
We’ve observed similar patterns with stories from the Philadelphia Inquirer, the Atlanta Journal Constitution, and others. Local news outlets can also be one of our greatest tools for correcting misinformation around voting incidents given their high level of trust and mandate to dig deep into issues affecting their constituents. The most prominent example of good correction work in our data comes from the work of the Milwaukee Journal Sentinel, who wrote a story stating that no absentee ballots were discovered in a widely-reported incident of discarded mail. Earlier that week, several partisan news outlets with national audiences, including Breitbart, The Gateway Pundit, and the Washington Examiner, had written stories about this incident strongly suggesting that absentee ballots had been thrown out in the mail. The Milwaukee Journal Sentinel’s correction became the single most retweeted story that we tracked for this entire incident, showing that a strong rebuttal of a misleading claim can receive even more exposure than the misleading claim itself.
Figure 6: A plot of the number of tweets and retweets linking to articles about ‘mail dumping’ in Wisconsin from four domains. Selected tweets with large numbers of retweets are highlighted, with arrows indicating the time of the tweet.
The Power of Small Accounts
The previous sections have focused on repeat offenders who create original content with high levels of engagement (i.e. retweets), but we also see that less influential, smaller accounts can also be repeat offenders, and in many cases to a more extreme degree. These less visible accounts (e.g. non-verified and generally with follower counts less than 10,000) are often amplifiers of misleading content. We found that 931 Twitter accounts have retweeted at least half of the 43 misleading stories in our data. Many repost the same stories multiple times. Some of these smaller accounts spread these narratives prolifically, tweeting up to 34 different misleading stories about voting.
These non-verified, repeat offenders form a consistent base for misleading voting stories being spread on Twitter. We found that just 10% of users are responsible for more than 50% of the retweets across our entire dataset, and 1% of users created approximately 15% of the retweets.
Self-identified Affiliation of Non-Verified, Repeat Offenders
Looking deeper into these smaller, non-verified accounts who repeatedly shared misleading election information, we were interested in how they categorize themselves through the use of hashtags in their Twitter bio. It is important to note that not all Twitter users add hashtags to their Twitter bios, and that those who do are not necessarily a representative sample of Twitter users — only 38.5% of our non-verified, frequent spreaders (those who have tweeted in at least 22 of the 43 separate issues the EIP has investigated) have at least one hashtag in their bio (out of around 27K).
Focusing on the 931 non-verified users who shared at least half of the 43 misleading stories investigated here, we consider the hashtags included in their Twitter bios (Table 2). While this is not a perfect measure, looking at bio hashtags can give some indication of the types of communities that these users identify with on Twitter. Arguably, these hashtags can indicate the type of followers that the accounts are trying to gain and share posts (including election related information) with. Among hashtags used by over 2% of these 931 users, the top hashtags appear to be primarily pro-Trump, pro-gun, pro-police, and QAnon-related.
As stated, this should not be taken as a definitive description of these users. This only represents a small fraction of the non-verified, repeat offenders and does not capture repeat offenders who do not use hashtags in their Twitter bio.
Table 2: Top hashtags appearing in the Twitter bios of non-verified, repeat offenders
Looking Forward
From this analysis, we can see that several domestic verified Twitter accounts have consistently amplified misinformation about the integrity of the upcoming election. These are often stories revolving around misleading narratives about mail-in ballots, destroyed or stolen ballots, officials interfering with election processes, misprinted or invalid ballots, and more. These “repeat offenders” often source misinformation from hyper-partisan media outlets, including the Gateway Pundit and Breitbart News. These media outlets often exploit local news outlets who report factual stories by mischaracterizing their stories and then elevating them to a wider network beyond what they are intended to push a delegitimization narrative.
Social media platforms have been taking action in removing or at least labeling content that can be misleading, but oftentimes this occurs after widespread dissemination of that content online. Platforms may need to begin enacting stronger sanctions to accounts and media-outlets who are repeat offenders of this type of misinformation. This could include labeling accounts who repeatedly share misleading information about voting or even removal from the platform. Labeling or removing just the content after virality may not be enough to curb the spread of misinformation on their platforms.