The Yield Optimization Dilemma

With Yahoo recently announcing their intention to join the rest of the civilized world in optimizing their text ads for yield (rather than for bids, which can be fairly meaningless in a PPC world[1]), and with Wall St shaking up a little, it's a good time to look into an interesting, and generally overlooked, aspect of the yield optimization algorithms used by Google, Yahoo and MSN.

But before jumping into the Yield Optimization Dilemma, a little bit of background:

First came Overture and ranked results by cost-per-click bid. The highest bidder always got 1st place, etc.

Then came Google, and by combination of: a) having to bypass Overture's patent on CPC-based ranking, and b) figuring out that CPC-ranking can, again, be quite meaningless[1], they introduced a yield-based algorithm for ranking ads.

Yield-based ranking basically means that ads are ranked according to the following formula:

Yield = CPC x CTR

Or in English - The yield of each ad is determined by what the advertiser is bidding (CPC) times the # of clicks users are actually committing on that ad.

The beauty of yield optimization is that it inherently improves the relevancy of the ads shown over time, and therefore is good for the publisher (more $$ money[2] for its screen real estate), good for the advertiser (ads shown where users "vote" them to be relevant), and obviously good for the ad network (again - more money).

On its surface, the yield optimization formula (Yield=CPCxCTR) has the feeling of being 'scientifically true', and can therefore always be applied to auction-based ad networks as-is.

But there's a devious little detail in these formulas that is completely overlooked these days, but could be a major issue in times of recession and diminishing advertising budgets. This factor, which is baked into all the yield optimization algorithms out there, can be summarized as:

The Yield Optimization Dilemma - When optimizing display of ads for potential yield, should it be the publisher's yield be optimized, or the ad network's yield?

I know - it's almost petty to mention this issue these days. Google is flush with ad dollars[2] and with multiple advertisers competing for every conceivable word mutation. With about $1B estimated of unspent ad budget by Google advertisers[3], Google can almost always show the best ad (=highest yielding) on all of its page views, both on owned properties (Google.com, Gmail, etc) and on 3rd parties via the AdSense network.

But if the advertising history has taught us one thing, it's that the ad market is cyclical and very sensitive to recessions. After every advertising bull run, it is almost guaranteed that a recession will kick in. And when that happens, the first budgets to be slashed are usually the advertising budgets.

When that day comes, and I bet it does[4], this is basically what will happen within the black boxes of Google, Yahoo and MSN millions of times per day:
"We have 1 ad with a remaining budget of $X which is the best yielding ad for keyword Y. That keyword has just been submitted by an AdSense/YPN/MSNwhatever partner, but we predict this keyword will be submitted to our own search engine (Google.com/Yahoo.com/Live.com) 100 times during the remainder of the day. Should we serve Great Ad to the partner site, or keep it for later for our property?"

Remember that every such decision is amplified about 4x by the revenue-share factor (=Google/Yahoo/MSN do not share revenue on clicks on their owned properties and therefore they make about 3-4x more on each click generated on those properties versus clicks on sites within their networks).

In numbers this is how this decision might look on 2 ads, one of which is yielding a $1 eCPM, and the other yielding $1.5 (assuming a rev-share of 70-30 with the publisher[5]):

1. Do-No-Evil algorithm (or - Good Ad served on partner site) - publisher makes an effective CPM of $1.05, while Google makes an effective CPM of $1.45
2. Do-Evil-As-Long-As-Nobody-Notices algorithm (or - Good Ad kept for Google properties) - Publisher will drop to a $0.70 eCPM while Google's eCPM jumps to $1.80

That's a 25% difference in revenues for the ad networks operating their own properties, all with a simple flick of an algorithm that only they control, and probably only they truly understand.

In times of advertising recession, when this Yield Optimization Dilemma will pop up on servers many many times a day, the decisions made may amount to tens or even hundreds of millions of dollars annually going (or more likely - NOT going) into publishers' pockets.

Publishers should be aware of this, as it's part of the cost of doing business when handing ad management over to companies that are big publishers themselves and have a huge financial interest in monetizing their own content before they monetize a partner's site.

 

[1] I say that CPC-based ranking can be meaningless, because an advertiser can bid $50 per click yet have an ad so irrelevant that no one ever clicks on it, making its overall yield for Yahoo and the publisher a nice round $0 for all the impressions it was shown on.

That $50 ad may be "pushing out" an ad with a 50c bid that's extremely relevant to the keyword and can get tons of clicks, making it a very high yielding ad.

[2] This one's for you, Tami.... ;-)

[3] As estimated by Goldman Sachs analyst Anthony Noto during Google's recent analyst day.

[4] Of course Google is much more resilient to such economic downturns, thanks both to it's large advertiser base (~350K?), and the fact that it is heavily geared towards direct marketers who are much less likely to slash ad spending than brand advertisers. But if I have to place my chips, I will bet that a recession will eventually hit, and even mighty Google will find itself with more content than it can supply ads and budgets for.

[5] This is obviously an extremely simplified example. There are many other parameters in the real world, but the genera idea is basically the same.

Comments

[Gab Goldenberg]

I seem to be extremely late to this party, but I only just discovered your blog and so on. I'd known about Outbrain for a bit, but after seeing your involvement with Zell, and the mention of Outbrain there... I eventually came full circle and found your blog, Yaron.

Anyways, there's a fundamental flaw with your problem as regards search engines. The following day is happening now, but will be increasingly less likely to happen over time as best practices grow mainstream.

"When that day comes, and I bet it does[4], this is basically what will happen within the black boxes of Google, Yahoo and MSN millions of times per day:
"We have 1 ad with a remaining budget of $X which is the best yielding ad for keyword Y. That keyword has just been submitted by an AdSense/YPN/MSNwhatever partner, but we predict this keyword will be submitted to our own search engine (Google.com/Yahoo.com/Live.com) 100 times during the remainder of the day. Should we serve Great Ad to the partner site, or keep it for later for our property?""

The best practice I'm referring to is having separate search and content network campaigns. The result is that daily budgets are separate for each and so there's a limited or negligible chance that such calculations may happen.

Where things get tricky is when the engines are also publishers, eg with Gmail and Youtube - though another growing best practice is usually to exclude them as low ROI sites.

On a side note, you may be itnerested to hear that affiliate networks approach this dilemma (e.g. capped amount of leads merchant X will buy per day during his trial period) is to offer the opportunity to affiliates who have the best relationships with their managers.