Since Elon Musk took over Twitter, the Tesla
TSLA,
CEO’s most visible changes to the social network have been small and cosmetic: changing the logo, changing the name, squeezing revenue out of the familiar blue checkmarks.
That’s largely because it’s hard to measure the big changes. Many users who celebrated Twitter before Musk’s arrival may now feel that X, formerly Twitter, is no longer the place for them, but can’t explain why.
Some users say that X is not moderating content and not paying attention to facts. Users are also reporting an increase in antisemitic and racist content, and many say the social media platform has “turned far more combative.”
To test if this subjective assessment is based on evidence or is simply anecdotal, we conducted a study. We collected Twitter data from before and after the completion of Musk’s purchase in October 2022 to test the hypothesis that the network is now less fact-based. Using data on engagement with posts, it’s possible to determine the voices that are dominating the conversation and the ones that are losing influence.
The data are clear: the face of X-Twitter is changing, and not for the better. X’s user base now is less interested in unbiased facts — accounts of fact-checkers and of unbiased media outlets are receiving 52% and 27% fewer interactions, respectively. Less trustworthy sources are receiving more attention — sites like The Joe Rogan Experience and Breitbart News, for example, increased their user engagement by 22%.
“We used Twitter’s Academic Research API to scrape 16 months of tweets from thousands of media outlets. ”
Let’s get specific on our study. In May 2023, we used Twitter’s Academic Research API to scrape 16 months of tweets from thousands of media outlets, covering the period from January 1, 2022, to April 30, 2023 (changes to the API since then make it hard to collect additional data). We focused on outlets with labels from MediaBias/FactCheck (MBFC), which classifies them into five political (least-biased, left bias, left-center bias, right-center bias, right bias) and four scientific categories (pro-science, conspiracy and pseudoscience, satire, questionable sources), based on their levels of potential bias and misinformation.
To measure engagement, the best metric to determine what stories are being amplified on the platform, we identified how frequently one tweet was @ or hashtagged (#) by Twitter users using the Twitter Count API Endpoint. For example, a tweet from Snopes — the fact-checking outlet — might generate 247 replies, responses and retweets, each of which constitutes an @ or #.
Out of the 4,844 media outlets listed by MBFC, we studied all 3,852 outlets that have Twitter accounts. We focus on comparing Quarter 1 in 2022 and Quarter 1 in 2023 in our analyses so that the unusual election period in November 2022 cannot explain the results; all results reported below remain even if we compare the entire pre- vs. post-Musk data.
We verified that the number of posts from average samples of these outlets did not change over the course of the study period, and we control for the number of posts in our analysis to ensure that it does not fully explain changes in engagement.
3 key takeaways
We highlight three main takeaways from our study that show how X-Twitter has fundamentally changed for users after the Musk purchase: On Elon Musk’s X, users are much less interested in fact-checker accounts, much less engaged with pro-science accounts, and much more engaged with extreme media outlets (and less engaged with the least-biased media outlets).
First, we found a large drop in engagement with fact-checkers and fact-checking sites like Snopes, PolitiFact, and OpenSecrets.org. Prior to Musk’s takeover in 2022, the average engagement with fact-checkers’ Twitter accounts was 76.9 per account per day (first-quarter of 2022). Following Musk’s acquisition a year later, in 2023, this figure dropped by 52% to 36.9 (first-quarter of 2023). More than half of the 13 fact-checkers experienced a significant decline in engagement levels, and eight showed a statistically significant decrease. Though the number of posts for these fact-checkers went down as well from 8.28 to 5.99 per account per day, this does not fully explain the effect on engagement with the posts.
Beyond fact-checkers, we also analyzed engagement with the media in general. Engagement with all 58 satire sites (such as The Onion) increased 32% to 82.6 per account, per day from an average of 62.7 (Q1 2022 vs. Q1 2023). Comparing the same two quarters, engagement with all 199 conspiracy and pseudoscience accounts (such as ZeroHedge and DoctorOz.com) remained unchanged, to 89.9 from 90.7. However, there was a drop in engagement with all 176 pro-science accounts (such as Nature, the World Health Organization, and the U.S. Centers for Disease Control & Prevention). Engagement with these accounts fell 13% to 104.7 in Q1 2023 from an average of 119.9 per account per day in Q1 2022.
Lastly, we found that users are engaging less with the most balanced sources when grouping media outlets according to their political stance. There’s increased engagement with all 216 right-biased media outlets (for example, The Daily Wire and Daily Telegraph), from an average of 197.8 per account per day to 349.4, a 77% increase, but no change in the 308 left-leaning media outlets (which remained at 363.1 on average).
At the same time, the data show decreased engagement with what MBFC labels moderately right-center bias outlets (496 outlets, such as the New York Post and The Wall Street Journal), from 175.1 to 151.4, a 14% drop, and 833 moderately left outlets (The New York Times, The Washington Post and Bloomberg News, for example), from 318.1 to 266.5, a 16% drop. Engagement with the 1081 outlets labeled least-biased (such as The Hill, Reuters, Sky News, and The Economist) fell to 80.1 from 110.4, a 27% drop.
“One possible reason why people are leaving X: more extreme bias.”
The change in conversation — an increase in engagement with more extreme elements and less engagement with fact-checkers and less-biased media — may be one reason why people are leaving the platform. An April 2021 poll — pre-Elon Musk — found that one-in-five U.S. adults, or 23%, with a higher prevalence among the youngest demographic (42% of adults ages 18 to 29), said they used Twitter. In September 2023, almost a year after Musk bought the platform, Variety reported that the monthly active users for X-Twitter dropped 15% worldwide and 18% in the U.S. People currently on Twitter are less likely to engage with unbiased media sources and fact-checks compared to people on Twitter prior to the Musk purchase.
X is in a transitionary period. It’s likely that it will never cease to exist, although advertisers pulling their dollars may precipitate its demise. As users change their relationship to the site, what once was a trusted public square becomes closer to a conspiracy-fueled hotbed for extremism.
Gita Johar is the Meyer Feldberg professor of Business at Columbia Business School and a former co-editor of the Journal of Consumer Research. Yu Ding is an assistant professor of marketing at the Stanford University Graduate School of Business.
More: Nullification of Elon Musk’s $56 billion pay package could be a wakeup call for corporate boards
Also read: Evidence EVs are a fading fad is ‘rolling in fast’ as Tesla, GM and Ford slash prices
