SearchTools Blog (searchtools) wrote,
SearchTools Blog

Search Indexing Crawler Tests

SearchTools Test Suite provides a suite of test pages for how well search indexing robots can handle robot rules and complex linking. Many robots are easily confused by anything beyond a simple URL, so these test help us identify the ones with more sophistication.

In addition, these tests will tell us how many robots can handle text in ALT and Comment tags, HTML header tags such as Meta Keywords, and more.

To try out this system, we've coded each page with "RTest", and more specifically, with "RTestGood", for successful indexing and "RTestProblem" for pages which should not be indexed.

NOTE: I know many of these tests are now funky, and have been since the site changed servers. I will be updating them as I have time and energy. Please give me suggestions and comments and offers of help by replying to this blog entry.

Indexing Test Suite

  • Post a new comment


    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.