Your SEO Plan, Week 4, Monday: Robots Visiting Your Site
In Chapter 7, “Month One: Kick It into Gear,” we discussed search engine indexing of
your landing pages. With a little sleuthing, you can determine which search engine
robots have visited your site and when.
To scope out robot visits to your site, do the following:
• First, find the name of the robot that you are interested in monitoring. Table 10.1
lists the robot identifiers for the four search engines we’ve featured in this book,
but be prepared for some variations.
Robot Identifiers for the Four Major Search Engines
• Then, review your website stats program. Look for a section called “User
Agents,” “Robots,” or “Browsers” (servers interpret a search engine robot as a
type of browser).
• Depending on the sophistication of your stats program, you may be able to spec-
ify a date range and view robot visits. You might even be able to see exactly
which pages within your site were visited.
If this all feels a bit too tedious, you may wish to look into software that pro-
vides simple yet detailed reports on robot visits. One such program is Robot Manager,
Your SEO Plan, Week 5, Tuesday: Google Sitemaps
In Chapter 8, “Month Two: Establish the Habit,” we talked about building a site map
page for your website. The Google Sitemap is a specially formatted “map” made just
for Google’s search engine spider to use as its very own key to navigating your site.
Google Sitemaps (in beta as of this writing) is especially helpful if Google has a
hard time indexing your site; for example, if your site has a very large number of pages
or dynamically generated pages. Here are the basics:
• You’ll first need to create a specially formatted file—a Sitemap—using Google’s
Sitemap generator tool. Google’s help page at
has the scoop on how to generate a Sitemap and
c10. 8:10 286