Bending the Rules
A typical situation when “bending the rules” may be useful is when a site already exists and presents a
flaw that cannot be overcome without a complete redesign. Usually a complete redesign, in the context
of a functioning web site, is a complex and arduous undertaking. At best, it cannot be done within the
time limits prescribed. At worst, it is completely impossible either due to budget or internal politics.
Perhaps the site is designed entirely in Flash (see Chapter 6), or it employs a URL-based session-handler
that could throw a spider into a spider-trap of circular, or infinite references. If a total application rewrite
is not an option — as is usually the case —
may be employed. Cloaking implies delivering different
content depending on whether the user agent is a human or a search engine spider. In the former case, an
HTML-based version of the site could be presented to the search engine spiders instead of the Flash ver-
sion. In the latter case, when the user agent is a spider, the site could use cloaking to remove the session
ID and other potentially confusing parameters from the URL, hence removing the spider-trap.
A well-known example of cloaking is that employed by the
New York Times
. Essentially, the
web site requests users to create (and pay for) an account with them for certain premium content —
as shown in Figure 8-1.
However, this restriction isn’t imposed on search engines. Indeed, the
New York Times
engines to browse and index its content without an account, which most probably gets
a lot of incoming traffic from search engines. A full write-up is available at
Chapter 8: Black Hat SEO
c08.qxd:c08 10:59 175