Web X.0

On search engines & Yellow Pages

Interesting (though far from surprising) news yesterday from Yahoo! on evolving Yahoo Local into a de-facto Yellow Pages platform. Original announcement here, coverage by TechCrunch here, and by John Battelle here.

This reminded me of a an email I sent to a colleague last year. The background was a growing number of requests from Yellow Pages (YP) companies to use our services to expose their specific YP listings on the major search engines[1]. My email below (slightly edited to maintain  confidentiality, etc) was part of the discussion around this direction. You can substitute the name I chose (YP-Co.) for mostly any traditional yellow pages player out there as anecdotal evidence suggests that mostly all of them are either considering or already feeding their listings into the major search engines:

From a strategic point of view, I think YP-Co is going to do a big mistake by feeding their listings into Google. The way I see it, they’re digging their own grave, for the sake of short term benefits. Here’s why:

Google is quickly taking over YP-Co’s market, and YP-Co is lending them a generous hand. By feeding their listings to Google, Google gets the local coverage they badly need, while building their own advertiser base in parallel. For Google this just primes their real future business and makes it easier for them to conquer the local space directly; for YP-Co this is a good way to get a temporary spike in traffic and then slowly decline as Google builds more and more direct relationships. I can’t see any long term strategic value for YP-Co coming from this deal. I also can’t see them sustaining a long term business where they’re an expensive middle-man that essentially provides a dumbed-down version of what Google offers.

In addition, YP-Co are essentially starting to flood the search engines with advertisers competing with YP-Co on the same keywords and inflating YP-Co’s own marketing expenses. At the end of the day, a click going to a YP-Co plumber page is 10x more valuable to them than a click going directly to a random plumber. The YP-Co click has residual value and over time gets people to understand there’s value in searching for services within YP-Co. With every click going directly to a random plumber, they’re enforcing the perception that Google is actually THE go-to place for service providers, and further obscuring YP-Co. So not only are they hurting themselves by obscuring their brand value, in the process of doing that they’re inflating their marketing budgets on very important channels (push even as little as 2-3 advertisers on a category, and the YP-Co ads are either going away, or will cost them a shitload of money… )

eBay is a good example for getting this – They spend tons of money on SEM (probably the biggest spender in the world), but always promote only their category pages and never ever ever promote a specific seller page. They seem to be extremely strict about this policy, as they understand that 10x value thing between their own page and a random seller’s page. YP-Co’s case is even more extreme, as the ‘random seller’s’  page on eBay is at least under the eBay brand and within their framework… with YP-Co the links just go off to an external random site.

In short, it seems to me like YP-Co is in a lame dog chase after Google, while trying to convince themselves that they’re making smart moves ahead of the curve.

This is of course a classic example of companies caught in the eye of the Aggregation Paradox. Unfortunately, most companies blinded by the chase after next quarter's numbers fall right into the Aggregation Paradox trap. And that's what makes the companies that get it (Google, Yahoo, etc) such wonderful businesses.

[1] Disclosures, disclaimers, clarifications and other vegetables:
A distinction should be made between two cases of promoting yellow pages (YP) content on search engines:

1. Promoting of the YP category pages on the search engines. Example here.
2. Promoting of specific listings (or - businesses) that are listed within the YP database.

While I sound skeptical about the 2nd type of SEM, I applaud YP companies for being smart and practicing the 1st type.

Beyond the obvious fact that this attracts highly qualified traffic, the 1st type of SEM actually increases the YP's brand and the importance of their site. A user reaching a category page on a YP site must still engage with the site (drill down, search, browse, whatever) in order to see the business listings. Next time s/he is looking for a business there are higher chances of him/her going directly to the YP site and skipping the (now) redundant search engines step. That is great for the YP and helps maintain a sustainable and even growing business.

It's the other type of promotion, that of specific business listings within the YP, that I am referring to in my mail and which I think is a slippery slope in the long term.

And to the disclosure - My company is involved in different ways in both flavors of YP promotion on search engines.

 

The Aggregation Paradox, Part 2a

Since posting yesterday about the major search engines avoiding being aggregated, I was looking for the right bottom line metaphor for describing the situation. Got it!:

A search engine preventing others from scraping its pages, is like a plagiarist suing someone for copying "his" writings.

 

Urrgghh! - Windows updates...

What's up with the forced Windows updates? Did your computer restart on its own last night too?!

It seems like MS assumes that forced restarts are OK to inflict as long as they properly save all un-saved documents. That may have been a safe assumption 10 years ago, but is a ridiculous assumption in 2006.

What's growingly important is all those windows that were simply open - browser windows, word documents I was reading, videos/podcasts I was in the middle of viewing/listening, etc, etc. It's knowing what was open, and in what state, that matters. Heck - Windows is not even 'aware' anymore of most of the un-saved docs I have at any given moment, because most of them are in browser windows which Windows never considers as un-saved.

It seems that Microsoft fails to understand that saving my open docs is solving probably 5% of what's needed before inflicting a restart. If you are going to hit me with a restart, make sure my computer is back to its identical state after the restart.

Thank god I at least have Firefox with the excellent must-have SessionSaver extension...

The Aggregation Paradox, Part 2

In a previous post I outlined the Aggregation Paradox and its ramifications on a variety of online businesses.

A curious fact is that one category of web companies has been able to completely avoid being aggregated. It's really quite fascinating, because this category includes the biggest aggregators of them all - the search engines.

Theoretically, a company with very little infrastructure could crawl Google's search result pages (SERP's) and provide an almost identical service to Google sans the costs involved with crawling and indexing the entire web. Moreover, the company could scrape and aggregate results from multiple search engines and provide a superior user experience with a fraction of the cost.

I know - this all sounds like the good old meta-search engines which have been around since, well, just about since search engines have been around. But meta-search never really took off, while aggregation in so many other categories has in a big way. Why is that? True - there are some technological barriers that make search trickier to aggregate than some of other categories (size of the index, dynamic blurbs, etc). But those are all solvable. So what really makes search aggregation different?

Simple - The search engines, being the mothers of all aggregators, know better than anyone else the Aggregation Paradox and the long term risks it imposes and those who get aggregated.

The engines have all jumped through hoops over the years to prevent anyone from aggregating them in a serious way. They've placed legal, economic and technical barriers to prevent aggregation. Here, for example, is a piece from Google's search T&C's:

No Automated Querying
You may not send automated queries of any sort to Google's system without express permission[1] in advance from Google. Note that "sending automated queries" includes, among other things:

• using any software which sends queries to Google to determine how a website or webpage "ranks" on Google for various queries;
• "meta-searching" Google; and
• performing "offline" searches on Google.

Please do not write to Google to request permission to "meta-search" Google for a research project, as such requests will not be granted.

Kind of an interesting policy for a company that's built entirely on NOT applying ANY of these restrictions to the sites that it crawls and aggregates. BTW - I can personally testify that this is probably one of Google's most enforced terms. Try to aggregate a few search result pages, and Google's legal team will come after you within minutes.

How do the search engines get away with this? Why isn't anyone calling them out?! The answer lies of course in the Aggregation Paradox, and in who gets it (Google, Yahoo, etc) and who doesn't (mostly everyone else that's addicted to the short term traffic...).

More on the Aggregation Paradox, and possible solutions to it, soon.

[1] "Express permission" = paying Google so much per query that a meta-search will find it very hard to earn anything...

Microsoft digs Israel (x2)

A couple of weeks ago I pointed out that Google was the first big internet company to set up an R&D shop in Israel. As I said:

The only real surprise here is that it took so long for any of the big internet companies to set up R&D shops in Israel in a serious way.

Not surprisingly, Microsoft announced today the opening of their 2nd R&D facility in Israel:

U.S. software giant Microsoft Corp. said on Wednesday it would open a new research and development centre in the Tel Aviv area in what will be its second such centre in Israel.

Microsoft Chief Executive Steve Ballmer said the Israeli R&D centres will play an increasingly important role in the company's global R&D strategy.

While I obviously have no idea what the new center will focus on, I have a strong feeling that the proximity to Google's announcements is no coincidence. I won't be surprised if we start seeing Live.com stuff coming out of Tel Aviv in the next few years.

Who's next? Yahoo?

Danny Sullivan totally misses tagging

In a recent post, Danny Sullivan goes ballistic about social tagging (such as del.icio.us) and its usefulness for improving search engines:

...we've had tagging on the web for going on 10 years, and the experience on the search side is that it can't be trusted.

The meta keywords tag has been around for nearly a decade. The idea behind it in part was that people could use the tag to classify what their pages are about, as well as provide copy that search engines could index if the relevant text for some reason couldn't fit on the page itself.

...In addition, none -- NONE! -- of these search engines now or ever has made use of the tag in a way to let you perhaps see all the pages "tagged" to be on a particular subject. Why not? The data is largely useless.

Mark me extremely dubious that tagging will make major inroads in improving search. And if I'm wrong, I'll happily mea cupla. But after 10 years of tagging, the experience so far gives me good reason to be dubious.

Danny misses the whole point. Meta-tags and 'bookmark tags'[1] like del.icio.us are totally different beasts. The difference is not, as Danny points out, the fact that meta-tags are self-provided tags and del.icio.us tags are not - that hardly matters. The huge difference is the basic underlying value proposition that each of these tagging types offers:

Meta-tags were designed primarily for exposing certain keywords to a search engine as a means for improving ranking on those keywords. That is their sole purpose and sole value. So I, as a creator of meta-tags for my site, am not 'awarded' for creating good meta-tags, or useful or relevant ones. I am awarded only by getting improved search rankings and increased traffic on those keywords I choose. There's no personal value gained by writing good meta-tags... on the contrary - if everyone else is doing keyword stuffing, I may drop in ranking if I write proper meta-tags.

This underlying design of meta-tags almost incentivizes misuse and is therefore fundamentally flawed. No wonder none of the search engines use a flawed system.

Bookmark tagging solutions like del.icio.us on the other hand provide a fundamentally different value proposition. As a del.icio.us tagger, I am organizing interesting pages for myself so that I can go back to them in the future and easily find them. I am doing it for pure egoistic reasons, and I have all the reasons in the world to do a good job of it. Sure - whatever tags I put on del.icio.us may have some grander value for someone else (meme trackers, search engines, etc). But frankly I don't really give a shit about those other uses when I'm tagging something. Therefore the underlying nature of social tagging apps (or 'bookmark tagging') incentivizes proper tagging[2].

Of course there could be attempts to abuse bookmark tags like del.icio.us if a significant financial incentive gets tied to their tags (in the form of using tags for search engine rankings).  But:

1. The more people use it for their own selfish reasons, the more difficult it is to abuse it. Think of this as sort of a Wikipedia... sure - some folks may try to abuse it once in a while and change entries, but the more people you have using it legitimately, the more difficult it is for abusers to misuse the system.
2. Abuse attempts are 100x easier to identify on a bookmark tagging service than they are on meta-tags or links because the usage patterns of a non-abuser (big variety of URL's, variety of tags, etc) are so different than an abusing pattern (stuffing of specific tags on specific sites). In the meta-tag world there is hardly any such differentiation, making it extremely difficult to sort the valid from the abusive.
   

Saying del.icio.us tags are necessarily crap because meta-tags are, is like saying all self-published sites (aka 'blogs') are crap just because self-published sites on Geocities were crap 10 years ago. What's the connection? Only the word 'tag' I guess... not much else.

I applaud Yahoo for foreseeing this and making one of the smartest moves in search in recent years (and I believe this was primarily a search move).

[1] The proper name for del.icio.us class applications is social bookmarking. However, for the purpose of this discussion, I think that bookmark tagging is probably a better name for these apps. While they do have a significant social aspect, the social piece has very little to do with the motivation people have in tagging on del.icio.us. The primary motivation is much more of a 'bookmark' motivation - 'how can I save something so that I can easily find it later?'.

[2] I've quoted Joshua Porter in the past, but this applies here as well:

From now on I’m going to call this idea the “Del.icio.us Lesson”. This is the lesson that personal value precedes network value: that selfish use comes before shared use. We’re seeing it more and more everyday in services like Del.icio.us, Flickr, and is an interesting aspect of networked applications. Even though we’re definitely benefitting from the value of networked software, we’re still not doing so unless the software is valuable to us on a personal level first.

Apologies...

I'm messing with some domain settings. Apologies for all the CSS and RSS mess today... bear with me for one more day until I sort this out.

BillMonk - highly recommended

Once in a while a web company comes along and does a fantastic job solving a very particular day-to-day issue. Not trying to become the next big web thing or to dethrone Google or anything like that. Just solving one issue and doing a great job of it.

My 2006 favorite in this category has got to be BillMonk. They explain the service better than I could:

Friends cover for each other all the time. We make it easy to manage roommate expenses, split restaurant bills, settle up after a group vacation, and record simple debts.

They've also recently added support for tracking stuff (books, DVD's, whatever) you borrow or lend:

Friends lend books, movies, and other stuff. Make us your librarian and we'll make sure you know where everything is. Friends can even share their collections and browse the group lending library!

They do it all with a great simple interface and have all the right features (SMS support for when you don't have access to a PC, shared bill support, etc, etc) with none of the useless features they could probably bloat it with. The only feature I couldn't find is the payment option!... I'd happily pay a few $'s a year to help support this great effort.

Go and take their product tour or sign up now.

TechCrunch has a more comprehensive review of BillMonk here.

The strange thing with Google glitches

Usually bugs or server problems are annoying things. Why is it that I get a kick out of them when they happen on Google apps? Is it only me??

Barring server problems[1], Google definitely got the calendar right. This one seems to have been developed by the Gmail/AdWords/Maps/Finance/Analytics Google and not by the Page Creator/Google Base/Web Accelerator/RSS Reader Google. It's a great product, though would be (will be?...) 10x more useful if they had full syncing with Outlook. I can't see too many folks maintaining two non-synced calendars to run their lives.

[1] You'd think that with a capex spending rate of ~$1.5B per year(!!) they'd be able to handle the "heavy lifting" of some simple calendar tasks, but it seems like server capacity issues are now almost built in to every Google product launch. Strange...

Jim Lavoie on Jimi Hendrix

I recently posted about Rite-Solutions and how it harvests ideas from its own employees. Chris Flanagan of the Business Innovation Factory (BIF) was kind enough to post a comment on this blog shortly after and point to the video of Jim Lavoie (Rite-Solutions CEO) speaking about these ideas at a BIF conference.

I saw the video and highly recommend it (though the first half is sort of skippable). Go see it here.

I wanted to post the transcript of the video but couldn't find one, so I'll contribute my own little transcript for one Jim Lavoie quote which I particularly liked:

Imagine Jimi Hendrix comes into a room and he's got a new idea for a song. He's standing in front of 6 fat white guys with their arms folded asking him if he looked at all the other music ever published in the world to see if his song is really unique. Or if he's done a cost analysis. Or if he's looked at doing it offshore because 'maybe we don't have to do it here'. And there's always the boss who doesn't even talk to Jimi. He asks the lawyer "Is there any risk?".

I call it 'when the inventor meets the preventers'. And that's what's really stopping innovation: white guys that haven't written a song in a lot of years that stop the innovator with stupid questions.

Every word set in stone... Design (or management for that matter) by committee is a bad bad thing.

Thanks for the comment Chris!