New York Times
clearly uses IP delivery–based cloaking, and does not pay attention to the user
agent. If that were not the case, users could spoof their user agent and get a free subscription. We tried
it. It doesn’t work.
Meanwhile, Matt Cutts of Google states “
Googlebot and other search engine bots can only crawl the free por-
tions that non-subscribed users can access. So, make sure that the free section includes meaty content that offers
.” Danny Sullivan, editor of SearchEngineWatch, states with regard to the
New York Times
“Do I think the NYT is spamming Google? No. Do I think they are cloaking? Yes. Do I think they
should be banned because Google itself warns against cloaking? No. I’ve long written that Google
guidelines on that are outdated. To be honest, it’s a pretty boring issue to revisit, very 2004. The
NYT is just the latest in a string of big companies showing that cloaking in and of itself isn’t
We don’t necessarily agree with this statement. Clicking a link in a SERP that gets you to a subscription
to users. Results of a search engine query are supposed to contain data relevant to what
the user has searched for. We leave the assessment to you.
Cloaking may be becoming more normatively acceptable recently, but it should still be avoided as the first
choice for solving a problem. Solutions that do not involve cloaking should be applied instead. Some of
this was previously discussed in Chapter 8. Cloaking is often suggested as a solution to preexisting sites
that use Flash or AJAX, and it is also the only way to implement indexed subscription-based content, as
New York Times
Examples of legitimate uses of cloaking — in our opinion — are demonstrated in this chapter.
Clearly, despite the fact that certain uses of cloaking are becoming accepted, it is still a risk, and it is likely
that if you are caught by Google for doing cloaking that it deems as unethical, your site will be banned.
fore, if you do not own an extremely popular site whose ban would elicit a public response, and
you’re not willing to take a risk, cloaking should be avoided — at least for Google. Minor changes are
probably safe, especially for Yahoo! and MSN, as implemented in the exercise that follows, where you
replace a figure with text.
It is, however, very difficult to define the meaning of “minor change,” and a ban may still occur in the
worst case. As aforementioned,
was reincluded in the index in a matter of days; how-
ever, in practice, for a smaller business, it would likely be much more devastating and take more time
to get reincluded after such a ban. The cloaking toolkit provided in this chapter allows you to cloak for
some search engines and not others. We cannot comment further, because it is a very complex set of
decisions. Use your own judgment.
Using the Meta Noarchive Tag
One problem that arises with cloaking is that the cache feature provided by most major search engines
would display the cloaked version to human visitors, instead of the version they are intended to see.
less to say, this is probably not what you want for several reasons — among them, that it conve-
niently indicates to your competitors that you are cloaking.
Chapter 11: Cloaking, Geo-Targeting, and IP Delivery
c11.qxd:c11 11:01 222