Search Engines: How Did We Get Here?

search engines

It seems strange to say but, for those old enough to remember, there was a time when internet searches were not dominated by Google. Prior to its arrival, the one time alpha and omega of search engines was Alta Vista (founded in 1995). One can still find Alta Vista (resurrected by Yahoo) bravely hanging on in a very lonely and unvisited corner of the internet, quietly giving competent search results via a rather bland and unappealing interface (it has a baby blue background that reminds one of the color of an unwanted sweater at Christmas). Perhaps if one has the time, go and enter a query for old times sake; it is akin to visiting a long forgotten relative in an old age home. The gesture is bound be appreciated and Alta Vista still has the power to tell a good story or two (although, you may feel like you are listening to Yahoo – which now receives primary and paid search results via Bing, which is in turn in owned by Microsoft).

Reading over the names of the now non-existent search engines that began life in the mid 1990s does illicit a certain sense of nostalgia, though without the passage of time usually required to stir these feelings. We are of course talking about “internet” years, which in a way mirror dog years (though dog years have remained fairly static and predictable). Does anyone still remember these one time players from the seminal days of the internet (some of the names could easily be mistaken for hair metal bands): Excite, Magellan, Snap, Direct Hit, Hot Bot. Some have soldiered on, others have been absorbed or have faded away into well deserved obscurity.

Today, as we all know, the dictatorial and tyrannical ruler of the internet, when it comes to searches, is none other than Google. Google started life as a research project by Larry Page and Sergey Brin in 1996. By 1998, Google had been incorporated as a privately held company. Today, Google has an Explicit Core Search share of 66.4% (more than four times that of its nearest competitor – Bing/Microsoft). Google is now a part of the scenery, like wood paneling in the basement. You don’t really like it, its kind of bland and dated, but you are too lazy to take it down and re-decorate as it serves its purpose so you tolerated it.

The other distant, but managing to get by, search engine that is nipping at Google’s heels is Bing. This Microsoft owned search engine is slowly gaining in stature, but still has a long way to go before it is truly relevant. Many prefer it to Google (perhaps more so out of spite), as it gives a wider range of results and is not as inclined to burden the user with advertisements or cookies. That said, Bing has failed to catch on. All one needs to do is look at their own analytics to see which search engine is driving traffic to their site. Bing is responsible, across the board, for a very small percentage of that traffic. Why? Google, for all its problems, still gives users the results they want and provides a feeling of familiarity (see wood paneling). It begs the question then, “what does Bing offer that Google doesn’t?” The answer, unfortunately for Bing, is not enough to cause one to switch. Google has a form of brand loyalty that cannot be trumped at the moment.

Bing, or better yet, Microsoft, in a desperate attempt at relevancy, tried a side-by-side comparison (Coke-Pepsi taste test, anyone?) and for all intents and purposes it failed. It wasn’t that Google provided by far the better results, it is just that Bing didn’t bring anything else to the table other than a vague sense of, occasional, equality. Even though Coke changed its recipe, but then wisely reverted back to what made it great, it still won the Pepsi challenge – hands down (such was its hold on public consciousness; and the fact that it was simply a better product). It is probably safe to say that Bing and Google will have Coke-Pepsi relationship for the foreseeable future, despite Google’s best attempts to annoy those most reliant on its search results by changing its “secret” recipe via the never-ending Panda and Penguin updates.

So what of Yahoo? Yes, it is still around and has refused to leave like the ubiquitous reveler who doesn’t know the party has ended. Oddly, we still begrudgingly acknowledge its existence as is evidenced by its Explicit Core Search share of 12.8%. Honestly, though, most Yahoo searches are probably done by accident. Yahoo’s behavior is even more bizarre in that they have kept the aforementioned Alta Vista afloat. Perhaps it is a write off for tax purposes or the beginning of a retirement home for irrelevant search engines.

So who are the other players left in the North American search engine wars? There has to be some RC Colas out there, right? In third place, with an Explicit Core Search share of only 3.2% is the Ask Network (originally know as Ask Jeeves – founded in 1996). Ask Jeeves, for the multitudes who will not recall, was the first search engine to employ what is known as “natural language” queries as opposed to the more terse syntax required by other search engines. Ask.com still uses this method and has also expanded it to include conversion, math and dictionary questions, which are really its forte. With the Ask Network’s recent purchase of About.com, it may see an increase in its search volume, but nothing to bother Google and Microsoft. It should also be noted that Ask.com receives paid results from Google.

Languishing in fourth, and talk about staying in the fridge past your best before date, is AOL, inc. Surely this can only be due to all those computers that came preloaded with it being turned on from time to time to see if they still work and if there are any harvest-able parts in them. Oh, how the mighty have fallen. The real problem with AOL was how restrictive it was. They were intent on keeping its users within its sphere of influence by directing them to approved sites and services. It was an early attempt at a “dumbing down” of the web for the masses, which thankfully failed. PCWorld magazine even awarded AOL the number one position in its top ten list of most annoying tech products on April 16th, 2007, for its practice of direct marketing. PCWorld claims that between 1993 to 2006 that AOL sent out over 1 billion AOL discs (most of which, according to PCWorld, ended up at their office).

Around the world, the search engine equation really isn’t much different. Google still holds the top spot by about a 7:1 ratio over its nearest competitor, Baidu.com. For those unfamiliar with Baidu, it is a search engine designed for websites, images and audio files that contain Chinese language content. Baidu has also created a Japanese language search engine, which only makes programmatic sense considering the written languages of China and Japan are ideographic and have a shared history. Out of all the other search engines mentioned, Baidu, based on the pervasiveness of Chinese languages (Mandarin and Cantonese) and culture, is the most likely to enjoy the biggest gains against Google globally.

Yahoo comes in at about an 11:1 ratio when compared to Google internationally; Microsoft sites come in at a 25:1 ratio. Rounding out the top five is the little known search engine known as Yandex. Yandex is a Russian owned internet company, which also owns the largest search engine in Russia. In addition, the Yandex site was voted the most popular website in Russia, too (which may or may not mean anything considering all the controversy around election fraud there and the fact that Yandex’s 40% market share in Russia is still second to Google). Yandex does have a presence in the USA as Yandex Labs, which is located in the San Francisco Bay Area. Before Yandex can really make a global impact it is going to have to become the dominant search engine choice in Russian speaking countries first; if not, expect continued marginality.

So, what does the future of the internet searches hold, well, for many it will be one lidless eye watching over all one does. But, If history has shown us anything, it is that no company, institution or government has been able to maintain a monopoly, and one day, without warning, the next young upstart(s) will come along and displace the wise old man of the web (currently Google). Hopefully Google will depart with more grace than some of its predecessors. Case in point, as of the writing of this article, US regulators are about to sue Google for using its search prowess to stifle competition and push up online advertising costs. Is this the first chink in Google’s armor that will open the door to the competition?

Credit: a big thanks to comScore for their invaluable help and information.