The Iffy Quotient is a metric for how much content from “Iffy” sites has been amplified on Facebook and Twitter.
Social media sites and search engines have become the de facto gatekeepers of public communication, a role once occupied by publishers and broadcasters. With this new role come public responsibilities, including limiting the spread of misinformation.
The Center for Social Media Responsibility at the University of Michigan School of Information has developed a way to measure the progress of media platforms at meeting their public responsibilities. We use the term “Iffy” to describe sites that frequently publish misinformation. It is a light-hearted way to acknowledge that our categorization of the sites is based on imprecise criteria and fallible human judgments. We are publishing a web-based dashboard that charts the Iffy Quotient since early 2016. The dashboard enables comparisons over time and between platforms. This report describes the calculation of the Iffy Quotient in detail, discusses some of its potential limitations, and analyzes some of the trends.
Ongoing classiﬁcation changes for sites
Changes to the ratings of news and information sites will have an effect on the Iffy Quotient. Those changes might include a site being newly rated for the first time or a site having its previous rating changed. Please see our Ongoing Classification Changes page to learn more and to see a running list of pertinent classification changes.
How many sites are unclassiﬁed?
NewsGuard and Media Bias/Fact Check have not investigated every site. Sites that do not appear on any of the NewsGuard or MBFC lists we classify as "Unknown"—neither "Iffy" nor "OK." If NewsGuard and MBFC don't keep up with new sites, over time we would mistakenly misclassify more and more of the truly Iffy sites as unknown. To check for that possibility, the graphs below track the fraction of unknown URLs.
Engagement-Weighted Iffy Quotient
As a supplementary analysis, we also compute an engagement-weighted version of the Iffy Quotient. Rather than treating all popular URLs as equal, we weight them by the estimated engagement scores that NewsWhip provides us. The denominator, then, is the sum of engagement scores for the top 5,000 URLs, and the numerator is the sum of engagement scores for those URLs that are from Iffy sites.
A comparison of the Iffy Quotient based on URL counts vs. engagement scores: