Iffy Quotient

The Iffy Quotient is a metric for how much content from “Iffy” sites has been amplified on Facebook and Twitter.

Please click on any datapoint on the graph to reveal that date's Daily Details page. (This can also be done with any of the graphs below.)

Social media sites and search engines have become the de facto gatekeepers of public communication, a role once occupied by publishers and broadcasters. With this new role come public responsibilities, including limiting the spread of misinformation.

The Center for Social Media Responsibility at the University of Michigan School of Information has developed a way to measure the progress of media platforms at meeting their public responsibilities. We use the term “Iffy” to describe sites that frequently publish misinformation. It is a light-hearted way to acknowledge that our categorization of the sites is based on imprecise criteria and fallible human judgments. We are publishing a web-based dashboard that charts the Iffy Quotient since early 2016. The dashboard enables comparisons over time and between platforms. This report describes the calculation of the Iffy Quotient in detail, discusses some of its potential limitations, and analyzes some of the trends.

Our Process

Ongoing classification changes for sites

Changes to the ratings of news and information sites will have an effect on the Iffy Quotient. Those changes might include a site being newly rated for the first time or a site having its previous rating changed. Our primary source for site ratings (classifications) is NewsGuard. For sites not rated by NewsGuard, we use site ratings from Media Bias/Fact Check when available.

How many sites are unclassified?

NewsGuard and Media Bias/Fact Check have not investigated every site. Sites that do not appear on any of the NewsGuard or MBFC lists we classify as "Unknown"—neither "Iffy" nor "OK." If NewsGuard and MBFC don't keep up with new sites, over time we would mistakenly misclassify more and more of the truly Iffy sites as unknown. To check for that possibility, the graphs below track the fraction of unknown URLs.

Engagement-Weighted Iffy Quotient

As a supplementary analysis, we also compute an engagement-weighted version of the Iffy Quotient. Rather than treating all popular URLs as equal, we weight them by the estimated engagement scores that NewsWhip provides us. The denominator, then, is the sum of engagement scores for the top 5,000 URLs, and the numerator is the sum of engagement scores for those URLs that are from Iffy sites.

A comparison of the Iffy Quotient based on URL counts vs. engagement scores:

Reports and Alerts

For some interpretations of how the Iffy Quotient has changed over time at Facebook and Twitter, download our White Paper.

To be alerted when there are meaningful changes in the Iffy Quotient for either Facebook or Twitter, join our mailing list.