Winners Don't Take All: Link Popularity for the Rest of Us

From Search Engine Watch

Though a small number of sites get the majority of inbound links and traffic, a new study reveals a previously unknown pattern of web page connectivity and shows how new, poorly connected sites can compete.

Over the past several years, search engines have increasingly relied on "link analysis" in addition to traditional information retrieval techniques to help determine relevance. Whereas traditional techniques look at factors like textual, semantic, and conceptual relationships between pages and queries, link analysis attempts to understand the structure of the web and the "social networks" that are formed when people link web pages to one another.

Google's PageRank algorithm is probably the best known (but least understood) approach to link analysis. In simple terms, PageRank is relatively straightforward. It simply counts the number of inbound links to a page, and then calculates the "importance" of those links by determining the "reputation" of the pages providing the links. The result is the PageRank, and Google calculates this value for every page in its index.

Google's increasing popularity and importance, as well as the adoption of link analysis techniques by other search engines, has led many webmasters to aggressively seek out links from other web sites, hoping to boost their link popularity and thereby boosting their position in Google and other search engine results.

The problem is, PageRank and other link analysis approaches tend to work well for large, well-established sites that the web community has overwhelmingly "voted" for by creating links to them. Link analysis techniques work less well for small, obscure, or especially, new sites with few links pointing to them.

For this reason, Google and other search engines do many other things beyond and apart from link analysis to calculate relevance -- a fact many webmasters lose sight of in their frenzy to gain as many inbound links to their site as possible. Efforts at "link spamming" are easily detected, and search engines are increasingly penalizing sites that are caught attempting to artificially inflate their link popularity.

The good news is that there's hope for smaller, less well connected sites, according to "Winners Don't Take All: Characterizing the Competition for Links on the Web," a new study from computer scientists at the NEC Research Institute who have spent years studying the structure and size of the web.

The researchers found that the distribution of links within specific categories doesn't follow the same pattern as the overall web. Previous studies have shown that the distribution of links to web sites approximates a "power law," where a tiny fraction of sites receive a hugely disproportionate share of links, and the vast majority of sites are essentially ignored.

The new study found that the "rich get richer" phenomenon enjoyed by large, popular web sites varies significantly across different categories and within online communities.

The scientists examined network structures for several subcategories of the web, including university homepages, newspaper, and scientist homepages, as well as several e-commerce categories such as publications, consumer electronics, entertainment, sports, and photographers. They found that the degree of "winners take all" behavior varied greatly.

The distribution was closest to a pure power law ("winners take all") for companies, newspapers, and publications. In contrast, the distribution for universities, scientists, and photographers was much less biased -- individual members of these communities fared much better.

"We were surprised to find such drastic differences for competition within individual communities, as compared to competition viewed across the entire web," said Dr. David Pennock of NEC Research Institute, the study's lead author.

Another interesting finding was that these network structures bear strong resemblances to real world networks, including research paper citations, movie actor collaborations and U.S. power grid connections.

Pennock noted that the dynamics of information dissemination online have the potential to alter competition and diversity in commerce and society, and that an increasing percentage of commerce and communication is occurring on the web over time.

What are the implications of the study for searchers and webmasters? For searchers, the key point seems to stay alert to an ongoing potential "narrowing" of search results, as well-connected sites become increasingly dominant over time. If you want to broaden your results to incorporate smaller, less connected sites, use a search engine like Teoma that actively seeks "native" communities relevant to your query.

For webmasters, the implications are less clear, though one point seems certain -- spend less time working on your overall, global link popularity, and focus more on building up strong connections in the natural "community" of sites that share a similar focus to your own.

While this tactic of "sleeping with the enemy" might seem dubious in the real world, the NEC study strongly suggests that it's one of the best ways to strengthen your presence in the universe of the web.

As you might imagine, the paper is dense with complex formulae and nearly impenetrable mathspeak. But it's well worth a read for anyone interested in how the structure of the web itself ends up affecting our search results, and what webmasters can do to compete against the "big guys" who tend to dominate search results and web page traffic.

How Search Phrases can Help Copywriters - And Your Visitors 

by Nick Usborne, Guest Writer 

Writing effectively for search engines isn't the same as writing for people.  Here's how to get copywriters excited and involved in the search engine optimization process. 

It isn't easy to get copywriters excited about including key words and phrases in your Web page text. 

Web page writers will often cringe when asked to include specific terms. How come? Because no writer likes to be asked to shoe-horn particular phrases into his or her text. So you end up with a conflict. 

Here is how that conflict will often unfold. 

Let's say you have a site devoted to advising people about how to advance their career or get a new job. 

The writer writes something like this: 

"Imagine having the following tools, resources, experts and services at your fingertips to help you manage your career and your life." 

Then the SEO expert says, "Can we include the phrase 'career planning' in that sentence please?" 

"You what?" asks the copywriter, all bristling and defensive. 

Like I said, no writer likes to be told what words or phrases to include in their copy. Copywriting is a creative process, and while every copywriter will work from a creative brief, they often won't take kindly to being handed a group of phrases to include and repeat. 

But here is one thing that decent online copywriters should be happy to do. 

Ask them to look at your site logs and study the phrases that your visitors typed in to various search engines when they were looking for your site. 

As regular Web users become a little more savvy about search engines, they type in fewer single words (too many results) and more complete phrases (better chance of finding what they want). 

So a search of the 'career planning' site logs might reveal phrases like this... 

* help writing a resume
* IT job listings
* career planning coaches and experts

And so on. 

Suddenly, instead of an 'SEO Expert' TELLING a writer which phrases to include, now the writer is listening to what site visitors are looking for. 

There's a big difference there. Key phrases are no longer being thrust upon a writer. That same writer is now doing personal research, based upon phrases that prospective customers are actually using. 

Few copywriters will argue with listening to site users. (If they do, that's a whole different problem!) 

So now you have a writer who is gladly including language and terms that are currently being used by your readers. 

Suddenly, everyone is winning. Your pages are becoming more search engine friendly. Your writers are becoming more familiar with what your visitors are looking for. And your visitors are finding highly relevant language and information on your pages. 

It's amazing how a potential negative, "Add these words and phrases", can so easily be turned into a positive. 

And what could be better than mining engine search phrases to better understand your own visitors? 

When you listen to your visitors, you are allowing them to become participants in the writing of your pages. And the more they participate, the more they will value the pages and the information that you deliver. 

Nick Usborne speaks, writes, and consults on strategic copy issues for business online. He is the author of the critically acclaimed book Net Words - Creating High-Impact Online Copy

Home Page

Sue Strand

605-274-1565 CST

sstrand at worldrecreationaldiscounts.com