Website Design Information

Optimizing Web Site Spiderability

Search engine spiders scan your Web site’s content by following the links that connect the pages on the site. Thus, when a spider visits a Web site, it will crawl through the site’s pages by following the hypertext links in each of the site’s pages until all of the pages have been read and considered for indexation. Broken or “uncrawlable” links will prevent the spider from visiting certain pages on the site. If, however, all the Web pages on a site are linked to each other with functional hyperlinks, a visiting spider will peruse every nook and cranny of the site. It is, therefore, of the utmost importance that all of your Web site’s internal hyperlinks are spiderable.

Most spiders still cannot read links that are built with Flash animation, image maps or JavaScript. Therefore, if the submitted Web site contains any such links, consider replacing or replicating those links with regular hyperlinks, so that search engines can read the links and thus spider the site.