Optimal Aggregation of Consumer Ratings: An Application to

Weijia Dai, Ginger Z. Jin, Jungmin Lee, Michael Luca

NBER Working Paper No. 18567
Issued in November 2012
NBER Program(s):   IO

Consumer review websites such as leverage the wisdom of the crowd, with each product being reviewed many times (some with more than 1000 reviews). Because of this, the way in which information is aggregated is a central decision faced by consumer review websites. Given a set of reviews, what is the optimal way to construct an average rating? We offer a structural approach to answering this question, allowing for (1) reviewers to vary in stringency (some reviewers tend to leave worse reviews on average) and accuracy (some reviewers are more erratic than others), (2) reviewers to be influenced by existing reviews, and (3) product quality to change over time. We apply this approach to reviews from to derive optimal ratings for each restaurant (in contrast with the arithmetic average displayed by Yelp). Because we have the history of reviews for each restaurant and many reviews left by each reviewer, we are able to identify these factors using variation in ratings within and across reviewers and restaurants. Using our estimated parameters, we construct optimal ratings for all restaurants on Yelp, and compare them to the arithmetic averages displayed by Yelp. As of the end of our sample, a conservative finding is that roughly 25-27% of restaurants are more than 0.15 stars away from the optimal rating, and 8-10% of restaurants are more than 0.25 stars away from the optimal rating. This suggests that large gains could be made by implementing optimal ratings. Much of the gains come from our method responding more quickly to changes in a restaurant's quality. Our algorithm can be flexibly applied to many different review settings.

You may purchase this paper on-line in .pdf format from ($5) for electronic delivery.

Information about Free Papers

You should expect a free download if you are a subscriber, a corporate associate of the NBER, a journalist, an employee of the U.S. federal government with a ".GOV" domain name, or a resident of nearly any developing country or transition economy.

If you usually get free papers at work/university but do not at home, you can either connect to your work VPN or proxy (if any) or elect to have a link to the paper emailed to your work email address below. The email address must be connected to a subscribing college, university, or other subscribing institution. Gmail and other free email addresses will not have access.


Machine-readable bibliographic record - MARC, RIS, BibTeX

Document Object Identifier (DOI): 10.3386/w18567

Users who downloaded this paper also downloaded these:
Mayzlin, Dover, and Chevalier w18340 Promotional Reviews: An Empirical Investigation of Online Review Manipulation
Celis, Lewis, Mobius, and Nazerzadeh w18590 Buy-it-now or Take-a-chance: Price Discrimination through Randomized Auctions
Levin w16852 The Economics of Internet Markets
Heldring and Robinson w18566 Colonialism and Economic Development in Africa
Netzer and Scheuer w18471 A Game Theoretic Foundation of Competitive Equilibria with Adverse Selection
NBER Videos

National Bureau of Economic Research, 1050 Massachusetts Ave., Cambridge, MA 02138; 617-868-3900; email:

Contact Us