Retrospective: How Google Monopolised The Search Market

Google November 1998
The main Google homepage as it appeared in 1998. Note the Yahoo!-style exclamation mark (which was dropped at the end of the site’s beta phase) and the Stanford Uni copyright attribution.

How strange it is to zip back through the years and revisit old articles from the 1990s – their writers referring to Google as the “The new search engine, Google.com”, and scratching their heads as to how the company was ever going to make any money. The scene, when Google first arrived, was awash with search brands such as AltaVista, Yahoo!, Lycos, Excite, Infoseek, WebCrawler, Hotbot, Dogpile, a host of others, and significantly as regards the future of Google, a new player by the name of GoTo.

Search results were very erratic at the time. Before Google, search engines could either opt for quality, or volume. Pre-sifting or filtering the results for quality (with human input) took time, and therefore limited the volume or size of a qualitatively-graded index. Going all-out for maximum volume, meanwhile, limited the quality of the results. So by and large, you could either have a smaller index with good relevance filtering, or you could have a much bigger index which was very hit and miss in terms of relevance.

But Google pioneered a groundbreaking search algorithm, which cunningly used the Internet’s own recommendation protocol – hyperlinks – to automatically determine the perceived quality of each piece of content. Only Google’s search system could create an index of massive volume, whilst inherently ordering all the contents based on perceived popularity.

Stanford University students Larry Page and Sergey Brin started the project in early 1996 as a reaction to their own disillusionment with conventional Web search. Their project’s basic core principle of auto-grading a Web page’s popularity based on the number and trust factor of hyperlinks directed towards it (a concept known as PageRank), was deceptively simple. It was, however, also world-beatingly powerful.

RELATED: Google – Limited Lifespan?
How the mighty Google monopolised the search market.

PORTAL HELL

When Google hit the World Wide Web in 1998, the default progression for a search company was to turn the search homepage into a portal. The industry was going portal mad. Everything either was a portal, or wanted to be a portal. If you built a search engine before the turn of the century, your goal was to ‘portalise’ it. That’s where the revenue was – or was supposed to be.

AltaVista Portal 2000
One-time search king AltaVista during its ‘portal’ phase in summer 2000.

But Google had other ideas. Indeed, the newcomer’s refusal to jump onto the portal bandwagon really confused some professional commentators. When Google categorically ruled out becoming a portal, some writers dismissed the company’s future on that precise basis. People knew Google had a great search engine from the start, but the notion was that if it couldn’t be monetised, it could not survive.

Google, meanwhile, could see the future. It could see that masses of people loathed portals, and that the tacky, distracting, media sideshows bloating out search pages were driving users away. Additionally, portals were getting harder to monetise, as a notion spread that they were less lucrative from the client side than had first been envisaged. The more bloat portals added, the less lucrative each individual placing was likely to become; and the less lucrative each placing became, the more bloat the portals needed to add. The portal concept was a vicious circle.

In overview, Web surfers wanted to find specific results, and then leave the search engine to explore those results. That was the whole point of a search engine. But the portals’ revenue stream needed users to stay on site, and that ran contrary to what a search engine was supposed to be for. It would make much more sense to implement a revenue stream which profited from users getting the best, most relevant search results, and then clicking on those results. In other words, a pay per click system, in which advertisers paid to have interested parties directed to their sites.

PAY PER CLICK

GoTo.com Homepage - 1998
GoTo.com (seen above in its original 1998 guise) was the forerunner of Google in terms of its minimalism and its preference for pay per click advertising. Indeed, GoTo’s subsequent owner would later litigate against Google in relation to a facet of AdWords, which it considered to be a patent infringement. Eventually, GoTo’s successful pay per click mechanism was acquired by Yahoo!

Pay per click had a rather vulgar air about it in the mid to late ‘90s. In 1998, the only competitor using the practice was the aforementioned significant newbie – GoTo. More established brands wouldn’t touch the idea, for fear it would taint their image and destroy them. But Google, itself part of the new wave, must have been taking note of GoTo’s early success with pay per click, and envisioning something closely related as the centrepiece of its own commercial needs. Pay per click was compatible both with clients’ needs, and importantly, users’ needs. It could be integrated into the search results themselves, which meant the perception of distractions and bloat would not enter the equation. Google’s concept of a simpler, wider-reaching, and more intelligent way of returning search results was tailor made for pay per click.

NO COMPETITION

Google After Full Launch in 1999
Google after full launch in the second half of 1999.

Fascinatingly, in 1999 when portal-mania was reaching a crescendo, Google not only flatly ruled out becoming a portal – it also saw itself as an entirely separate market from the portals, which it viewed as media sites with a token search box, rather than search sites with added media. Even in the first half of ’99 Google literally saw itself as having no competition, and that sounded very much like a genuine belief as opposed to arrogant sales speak. The claim also turned out to be entirely correct.

This prophetic post from December 1998 sums the situation up…

That a couple of grad students could build a better search engine than a whole raft of media and technology companies with stock-market valuations in the billions does not speak well of how these firms are spending their budgets.

But that author’s view of the overall picture was not shared by the IT media as a whole. At a time when Google had still not revealed its commercial strategy, and was assumed not to have one, there was a sense in many quarters that monetisation should come first. Some writers played down Google’s technical superiority – warning that it would count for nothing if the company could not find a means to earn serious revenue.

Whilst this debate was raging in mid 1999, Google acquired its first $25million in venture capital investment. Already, the secret plan was almost certainly a rough vision of the pay per click system AdWords, which officially launched a little over a year later. But a lot of people were underestimating the disarmingly coy Google in ‘99. It was just another of the many dot-coms in which people were investing “silly money”, seemingly without any idea of where the return would come from. That investment was a last chance warning sign for all Google’s competitors to take the company much more seriously, but none of them saw what was coming. They were too busy scoring own goals.

PUBLIC ADOPTION

Google Beta Screen May 1999
Google’s ‘BETA’ screen – May 1999.

If you search for genuine original Web posts mentioning Google in 1998 and 1999, there’s actually very little there. Virtually all of what you get by default is more recent posts which people have backdated. You have to heavily filter the results with advanced search techniques to drill through to the real stuff.

After you’ve singled out the posts that genuinely do come from the 1990s, much of what you have left is either heavy academic analysis packed with mathematical equations, or the odd professional appraisal. Indeed, for a sizeable chunk of 1999, Google’s homepage was still marked with a ‘BETA’ designation, and the service didn’t officially come out of beta for a full launch until 21st September that year. By then, in total the project had been on the go for the best part of four years.

Going into the new century, however, wider public discussion about Google did quickly and sharply increase. Even as late as 2000, some professional writers (or at least magazine contributors) were not necessarily awarding Google the status of top search engine in shootouts. But in real homes and offices, the exodus from AltaVista and Yahoo! towards Google was well in motion by 2000, and whilst some IT journalists still didn’t quite seem ready to admit it, the die was now cast.

Google did deliver better search results, but not, for most users, so obviously better that this factor alone could dismantle the competition. What really accelerated the migration into the arms of Google, was the chaos and bloat of previous search king AltaVista. AV’s whacking great page of portal features cheesed users off. You see that confirmed again and again in discussions from the time. AltaVista dropped its portal features in September 2000 (notably a short time after Google finally showed its commercial hand with the introduction of AdWords), but by then it was too late, and Google had already showcased its talents to AV’s disillusioned masses.

By winter 2001, the majority of ordinary users were acknowledging Google as the new king of search engines.

AltaVista December 1996
AltaVista circa December 1996. Five years later this hugely popular search engine had been well and truly turned over by Page and Brin’s almighty successor.

So it could be said that rather than Google striking a killer punch to the temple of AltaVista and Co, AltaVista and Co committed commercial suicide. The lesson was: don’t annoy your userbase with irritating crap when there’s a free, impeccably-behaved alternative on the scene. But it wasn’t a lesson AltaVista could learn for next time. There wasn’t going to be a second chance. Rivals needed to take Google seriously in the first half of 1999, when they still had the clout to shut it out of the mass market. They didn’t, and they paid the price.

ICING ON THE CAKE

Whereas a lot of Google’s late ‘90s rivals would knowingly damage the user experience in order to make more money, Google recognised that the user experience was absolutely central to the site’s success. One of the things we perhaps overlook today is the issue of page load times. With people using dial-up connections in the 1990s and the start of the new century, a page that took ages to load, really did take ages to load! Google’s minimalistic approach was not only easier on the eye – the fast page load was a timesaver for many, many users. And given the way ISPs charged for bandwidth back then, Google was also a money saver.

Another of the unsung features which helped Google kill off its rivals was the highly controversial cache. Google must have known the cache (which temporarily archived complete web pages on Google’s own site) was a massive risk in terms of potential copyright infringement cases, but it also knew the cache was an exceptionally powerful feature. Once again, when you look back at the comments of early Google adopters, you see the cache making a positive impression. Being able to access a search result even when the target site was down, at a time when sites were often down, gave Google exceptional reliability.

CONCLUSION

Google 2000 Adwords
Google around the “point of no return” in 2000, complete with an intro link for AdWords.

So Google’s package was very much about a compound effect, in which not just the main feature was better than the competition – but every detail was better. Whereas the competitors set out to persuade users that they wanted things they really didn’t want, Google just set out to give users what they did want. Why was that a better plan? Because persuasion costs a vast amount of money in ad campaigns, whereas stuff people automatically like will promote itself. From the start, Google intended to be as minimal with promotion as with its interface. But it didn’t matter. When you’re the best in every way, and your service is free, you don’t need to advertise.

The point of no return in the Web search market probably came in mid 2000, when after a string of awards, Google outgrew its rivals in terms of index volume, and was selected to power Yahoo!’s well established search engine. From that point on, there was a sense that the only tactic other huge search brands could adopt, was a resigned “if you can’t beat ‘em, join ‘em” policy. Everyone could now see that the mighty Google was unbeatable, and as its market share grew exponentially, it wasn’t just a better choice for paying advertisers – it was, realistically, the only choice. What’s the point of advertising on a search engine 98% of surfers don’t use? None. Advertisers will always want access to the majority, and that’s what Google has consistently provided ever since.

  Author: Bob Leggitt

Advertisements