Video aggregation

Dabble_logo Take video, the hottest thing on the web right now, and layer it with aggregation, which is a far better business than any single video source can ever be, and you get Dabble - a new meta-search engine for video content, aggregating results from YouTube, MetaCafe, etc.

All the advantages of a video site, with more variety and waaay less serving costs. Sweet!

I should have seen this one coming!...

More coverage on Mashable and TechCrunch.

On search engines & Yellow Pages

Yellow_pages Interesting (though far from surprising) news yesterday from Yahoo! on evolving Yahoo Local into a de-facto Yellow Pages platform. Original announcement here, coverage by TechCrunch here, and by John Battelle here.

This reminded me of a an email I sent to a colleague last year. The background was a growing number of requests from Yellow Pages (YP) companies to use our services to expose their specific YP listings on the major search engines[1]. My email below (slightly edited to maintain  confidentiality, etc) was part of the discussion around this direction. You can substitute the name I chose (YP-Co.) for mostly any traditional yellow pages player out there as anecdotal evidence suggests that mostly all of them are either considering or already feeding their listings into the major search engines:

From a strategic point of view, I think YP-Co is going to do a big mistake by feeding their listings into Google. The way I see it, they’re digging their own grave, for the sake of short term benefits. Here’s why:

Google is quickly taking over YP-Co’s market, and YP-Co is lending them a generous hand. By feeding their listings to Google, Google gets the local coverage they badly need, while building their own advertiser base in parallel. For Google this just primes their real future business and makes it easier for them to conquer the local space directly; for YP-Co this is a good way to get a temporary spike in traffic and then slowly decline as Google builds more and more direct relationships. I can’t see any long term strategic value for YP-Co coming from this deal. I also can’t see them sustaining a long term business where they’re an expensive middle-man that essentially provides a dumbed-down version of what Google offers.

In addition, YP-Co are essentially starting to flood the search engines with advertisers competing with YP-Co on the same keywords and inflating YP-Co’s own marketing expenses. At the end of the day, a click going to a YP-Co plumber page is 10x more valuable to them than a click going directly to a random plumber. The YP-Co click has residual value and over time gets people to understand there’s value in searching for services within YP-Co. With every click going directly to a random plumber, they’re enforcing the perception that Google is actually THE go-to place for service providers, and further obscuring YP-Co. So not only are they hurting themselves by obscuring their brand value, in the process of doing that they’re inflating their marketing budgets on very important channels (push even as little as 2-3 advertisers on a category, and the YP-Co ads are either going away, or will cost them a shitload of money… )

eBay is a good example for getting this – They spend tons of money on SEM (probably the biggest spender in the world), but always promote only their category pages and never ever ever promote a specific seller page. They seem to be extremely strict about this policy, as they understand that 10x value thing between their own page and a random seller’s page. YP-Co’s case is even more extreme, as the ‘random seller’s’  page on eBay is at least under the eBay brand and within their framework… with YP-Co the links just go off to an external random site.

In short, it seems to me like YP-Co is in a lame dog chase after Google, while trying to convince themselves that they’re making smart moves ahead of the curve.

This is of course a classic example of companies caught in the eye of the Aggregation Paradox. Unfortunately, most companies blinded by the chase after next quarter's numbers fall right into the Aggregation Paradox trap. And that's what makes the companies that get it (Google, Yahoo, etc) such wonderful businesses.

[1] Disclosures, disclaimers, clarifications and other vegetables:
A distinction should be made between two cases of promoting yellow pages (YP) content on search engines:

  1. Promoting of the YP category pages on the search engines. Example here.
  2. Promoting of specific listings (or - businesses) that are listed within the YP database.

While I sound skeptical about the 2nd type of SEM, I applaud YP companies for being smart and practicing the 1st type.

Beyond the obvious fact that this attracts highly qualified traffic, the 1st type of SEM actually increases the YP's brand and the importance of their site. A user reaching a category page on a YP site must still engage with the site (drill down, search, browse, whatever) in order to see the business listings. Next time s/he is looking for a business there are higher chances of him/her going directly to the YP site and skipping the (now) redundant search engines step. That is great for the YP and helps maintain a sustainable and even growing business.

It's the other type of promotion, that of specific business listings within the YP, that I am referring to in my mail and which I think is a slippery slope in the long term.

And to the disclosure - My company is involved in different ways in both flavors of YP promotion on search engines.

 

The Aggregation Paradox, Part 2a

Since posting yesterday about the major search engines avoiding being aggregated, I was looking for the right bottom line metaphor for describing the situation. Got it!:

A search engine preventing others from scraping its pages, is like a plagiarist suing someone for copying "his" writings.


 

The Aggregation Paradox, Part 2

In a previous post I outlined the Aggregation Paradox and its ramifications on a variety of online businesses.

A curious fact is that one category of web companies has been able to completely avoid being aggregated. It's really quite fascinating, because this category includes the biggest aggregators of them all - the search engines.

Theoretically, a company with very little infrastructure could crawl Google's search result pages (SERP's) and provide an almost identical service to Google sans the costs involved with crawling and indexing the entire web. Moreover, the company could scrape and aggregate results from multiple search engines and provide a superior user experience with a fraction of the cost.

I know - this all sounds like the good old meta-search engines which have been around since, well, just about since search engines have been around. But meta-search never really took off, while aggregation in so many other categories has in a big way. Why is that? True - there are some technological barriers that make search trickier to aggregate than some of other categories (size of the index, dynamic blurbs, etc). But those are all solvable. So what really makes search aggregation different?

Simple - The search engines, being the mothers of all aggregators, know better than anyone else the Aggregation Paradox and the long term risks it imposes and those who get aggregated.

The engines have all jumped through hoops over the years to prevent anyone from aggregating them in a serious way. They've placed legal, economic and technical barriers to prevent aggregation. Here, for example, is a piece from Google's search T&C's:

No Automated Querying
You may not send automated queries of any sort to Google's system without express permission[1] in advance from Google. Note that "sending automated queries" includes, among other things:

  • using any software which sends queries to Google to determine how a website or webpage "ranks" on Google for various queries;
  • "meta-searching" Google; and
  • performing "offline" searches on Google.

Please do not write to Google to request permission to "meta-search" Google for a research project, as such requests will not be granted.

Kind of an interesting policy for a company that's built entirely on NOT applying ANY of these restrictions to the sites that it crawls and aggregates. BTW - I can personally testify that this is probably one of Google's most enforced terms. Try to aggregate a few search result pages, and Google's legal team will come after you within minutes.

How do the search engines get away with this? Why isn't anyone calling them out?! The answer lies of course in the Aggregation Paradox, and in who gets it (Google, Yahoo, etc) and who doesn't (mostly everyone else that's addicted to the short term traffic...).

More on the Aggregation Paradox, and possible solutions to it, soon.

[1] "Express permission" = paying Google so much per query that a meta-search will find it very hard to earn anything...

The Aggregation Paradox

With the incredible ease of data aggregation that the web has introduced, an increasing number companies are now involved in the "Aggregation Paradox".

In this aggregation world, a company is either:

  • A Source - Sites that are creators of original content, whether it's news articles, real estate listings, job postings, etc.
  • An Aggregator - A site that aggregates original content from multiple Sources, and combines all those sources into a single user experience.

A few examples of these relationships:

The dynamics of the Aggregation Paradox:

  • Sources are eager to attract users, regardless of who sends them over and how they do it. If the entity sending them traffic isn't charging them anything - the Sources become even more ignorant to the entity handing them traffic.
  • The Aggregator's value proposition to the user is almost always better than that of any single source it aggregates. Why search 1 source -- the aggregator rightly claims -- when you can find data from 100 sources at once?
  • The content production costs fall almost exclusively on the Sources. This means that aggregators can be extremely flexible with how they monetize their service. For example, sticking a couple of AdSense ads around the aggregated content can usually create a very healthy and high-margin business for the Aggregator. This is a luxury the Sources hardly have due to the high production costs involved in creating the content.
  • The more Sources are aggregated into an Aggregator, the less significant each Source becomes. For example, a news aggregator that has only one Source for financial news will be severely affected if that source pulled out of aggregation. But if the Aggregator has 100 financial news Sources, the dropping of one Source would be practically an un-noticeable event.

 

Combine all of the above and we get the Law of Aggregation:

Galai's 1st Law of Aggregation: A Source that's being aggregated will see short term benefit while ceding long-term power to the Aggregator.

 

...which leads to the Aggregation Paradox:

The Aggregation Paradox - If a Source is aggregated, the Aggregator will prevail. If a Source is NOT aggregated, it doesn't matter and the Aggregator will prevail. 

 

I don't envy Sources having to deal with the dilemma that the Aggregation Paradox introduces to their business. And with Google becoming very aggressive as an aggregator (shopping, movies, autos and travel just to name a few) many companies that serve as Sources are in real jeopardy of going out of business if they don't quickly become aware of the Aggregation Paradox and its implications to their business.

Fortunately, not all is doomed for Sources. I'll cover all the options at their disposal for dealing with the Aggregation Paradox in a followup post.

Disclosure: Chicago Tribune and Realtor.com are both clients of Quigo which I have founded and where I am currently employed. I highly recommend clicking those two links and enjoying their wonderful news and real-estate services.

My favorite blogs

MyWay badge

Subscribers

Misc

  • Disclaimer
    ~~This is my personal blog, and any opinions expressed herein are mine and mine alone. Quigo and outbrain, my employers, are not responsible for anything I write, comments posted, or anything else in Web X.0 blog.
Blog powered by TypePad