Advertising Works - According to Yahoo! Research

We have discussed the possibility of evaluating adevertising campaigns on this blog before (here and here). So it is interesting to read that Yahoo! Research is measuring the effects of advertising on sales through a controlled experiment (more details available here). Researcher David Reiley has recently collaborated with Yahoo!’s Marketing Insights team in their ongoing efforts to help advertisers evaluate the effectiveness of their advertising campaigns.

Reiley joined Yahoo! Research because of his longstanding interest in field experiments and he points out a potential weakness of a study reported in a recent Harvard Business Review article, which measured large increases in sales due to online advertising. The study used large quantities of data from comScore, a key online information provider that logs the Internet browsing behaviour of two million users worldwide. By comparing the purchases of those who saw a given online ad with the purchases of those who do did not see it, the study concluded that there are large positive effects of online advertising.

However, "the population of people who sees a particular ad may be very different from the population who does not see the same ad,” says Reiley. Reiley's research made use of a database match between Yahoo! and a nationwide retailer by identifying users who registered the same email address with both companies. After finding over one million matched users, the researchers randomly assigned them to treatment and control groups for one of the retailer’s online advertising campaigns.

The project then tracked sales each week at the retailer, both online and in stores. In a paper co-authored with summer intern and MIT PhD student Randall Lewis, Reiley found that the online display advertising increased total revenues by approximately 5% for those users exposed to the ads, with 93% of the total effect happening in offline sales. They also observed online ads to have a large impact on sales even when the ads are not clicked: 78% of the increase in sales came from those who viewed, but did not click, the ads.

Could this have implications for the "pay-per-click" model in online advertising?