The World According To Google
Money from Google's IPO, meanwhile, will accelerate its continual upgrades of hardware and software and fuel new lines of business, such as a groundbreaking free Web-mail service announced last month. Gmail, as it's called, will offer consumers a gigabyte of free mail storage, many times more than the Yahoo! and Microsoft Web mail services. The catch: Subscribers have to endure E-mail ads targeted to their interests--as determined by Google's spiders, which will automatically scan every message. The proposed service has met with stiff criticism from some privacy advocates, but it plays to Google's supremacy in running computer systems that scan huge amounts of data and tailor advertising to the individual user.
For now, Google's dominance makes it the main target for Web site "optimizers": an army of consultants who try to reverse-engineer the ranking formulas and push a particular site to the top. Pages can gain rank legitimately by, for example, posting informative content, such as how-to instructions, that attract links from other sites. Less scrupulous methods include "cloaking," or presenting one page to users and another full of spider-friendly keywords to Web crawlers, and "link farms" of pages that exist only to point to a target site and boost its ranking. "We try to put those sites back in their place," says Peter Norvig, Google's director of search quality.
But changes in Google's ranking rules can also mean upheaval for businesses that have come by their rankings honestly. After last November's update, "I had people come to my booth and literally break down crying over lost rankings," says Bruce Clay, a longtime search-engine consultant. Webmasters greet such updates with the same enthusiasm as nasty storms and give them names reminiscent of hurricanes; November's became "Florida."
Google also referees noncommercial sites. A study coauthored by Zittrain at Harvard's Berkman Center for In-ter-net and Society found that the search giant, apparently at the demand of German authorities, expunges neo-Nazi sites from its German-language version. It also removed links to sites that the Church of Scientology accused of illegally posting its copyrighted material. But Google will not remove sites unless the complaints carry legal weight, and in each case it refers users to the freedom-of-information site chillingeffects.org, which posts the legal notices themselves, along with the questionable Web addresses.
The Web world may be in good hands with Google, says Zittrain, but it bothers him that only Google knows for sure. He believes search engines could reveal enough about how they rank sites to satisfy public concerns without giving away commercial secrets. "They wouldn't have to give up the recipe itself."
Winners and losers. But some critics do worry about Google's recipe itself--the PageRank algorithm. "It turns knowledge into a popularity contest," says NYU's Vaidhyanathan. And once a site ends up high in the rankings, it attracts new links and becomes harder to unseat, reinforcing the established hierarchy. That conservatism can make it hard for new sites, offbeat ideas, and minority views to find their way onto the first few screens of search results.