Wednesday: Robot Walk-Through
You’re all dressed up and the hors d’oeuvres are on the table. But is there a big Do Not
Enter sign on your door? You know the basics of how the robots find your site, and
you know whether or not your landing pages are indexed. Today you’ll knock down
any barriers that exist between the robots and your landing pages. And even if all of
your landing pages are already indexed, today you’ll learn more about keeping robots
happy so they’ll always stick around for the toast.
Take a look at your Baseline Monthly Report or your Rank Tracking Worksheet
to determine whether any of your landing pages are not indexed. Here are several rea-
sons a robot might not be reaching your landing page and possible ways to fix the
Robots can’t follow your links.
This could be as simple as a having no links from your
home page or your main site navigation to one or all of your landing pages. Or maybe
pull-down menus or pop-up scripts. Often, this is an easy fix: just add standard HTML
text links from anywhere on your home page to your landing page. (You’ll probably
want a site map as well. We’ll cover that next month.)
No stand-alone landing pages.
Maybe your site was developed in Flash, in which
case it really isn’t a group of individual pages but is rather one big file that’s hard or
impossible for the search engines to index as separate pages. Perhaps your landing
pages are generated dynamically or they show up only when a form is submitted or a
login ID is entered. Since robots don’t fill out forms or submit login data, they won’t
find these pages. Or, perhaps your landing pages are built in
, which means dif-
ferent elements of the page are broken apart into different URLs, and it’s impossible to
link to any one page individually.
In the long run, we’d like to see your entire site revamped as much as necessary to
get robots nibbling at every little crumb, which may mean a full overhaul of your site
structure. For today, our priority is getting those landing pages indexed! That means at
the very least rebuilding your landing pages as completely separate, linkable URLs. If
your landing pages are built in frames—Achooo! There’s a lot of dust on this website!
Get ready to have them rebuilt with their own URLs.
Your site asks too much from the visitor.
If the queen came to visit, you wouldn’t turn
her away if she wasn’t wearing the right hat. Treat your spiders the same way! Some
cookies? You get the point. You’ll need to eliminate these requirements on your landing
pages. If you’re not sure what your site requires, you’ll get a better sense of it when
c07. 8:08 156