December 19th, 2006

Eight Principles for Good Search Suggestions

Search Suggestion systems complement search engine results with human judgement, providing links to specific landing pages for common search terms where the algorithmic results may be confusing or unavailable. (For more information, see my Search Suggestions analysis).

This is a list of eight principles to keep in mind when administering a search suggestions system.

  1. Always work from real data. Do not guess what people are searching for, use the logs to find the terms, then search for the terms and look for the search results. If they're good, just stop!
    One site I worked on had recommendation terms four and five words long, and never actually typed by real people -- marketing department wishful thinking. It's better to concentrate on the top query terms, especially those with ambiguous meanings.
  2. Use Suggestions for content that's not accessible to your search engine. had many queries for Nine West shoes, but they don't carry that brand on the site. Their suggestion is to check a local store, and they also display other similar shoes as a search result
  3. Be sure the search engine recommendation module logs clicks so you can analyze the traffic -- are the Suggestions used or ignored?
    It's worth putting in the effort to understand how your Suggestions are being used. How many times are they clicked? What search terms lead to recommendation clicks? My rule of thumb is that if a popular search term leads to a recommendation click 30% of the time or more, it's working really well. The percentage of clicks may be much higher, if the recommendation is for content that's otherwise unavailable.
  4. Limit the number of terms and URLs. You want to avoid a management problem, because it's hard to keep track of terms, and the URLs will change.
    I've seen large sites with anywhere from 50 to over 1,000 recommendation terms. Stay on the low end of that range: if there are too many terms, either someone's gone overboard or the search engine itself is broken.
  5. Understand the Suggestions system: does it do stemming, substring match or just exact match? In many cases, it will be exact match only, which means you do need to enter plural terms when appropriate.
    Test what happens if the searcher enters two words, such as chocolate malt -- will it match Suggestions for chocolate, for malt, or only chocolate malt, in that order? What if they enter choco -- will it match Suggestions for chocolate malt?
  6. Try to match the suggestion text with search terms: the better the match, the more the user will notice it, understand it and click on it (if appropriate).
    I noticed one recommendation wasn't getting many clicks, though the search term was quite common, and technical enough that the search results were hard to understand. So, taking a leaf from Information Foraging theory and what little marketing I've picked up, I changed the link to both match the search term and explain what it meant. That did the trick - people clicked on it a lot more often because they felt confident in what they were going to get.
  7. Test with the top queries -- look for good results and unintended consequences
    Defining the top queries is tricky (and it will be another blog entry), but start off with the top 25 queries, do the search with the Suggestions enabled, and see what happens.
  8. Keep your Suggestions current -- create get some kind of script to check that the target URLs are still good, and run it at least once a month.
    There's nothing more confidence-destroying than clicking on a recommendation and getting an error message. Having an automated script means much less boring clicking, and if you use something like Perl's LWP library, you can compare the url and the final url, which may be a redirect.

I don't mean to impugn Search Suggestions here -- I think they're a fabulous solution to a problem of ambiguity and valuable pages being swamped in search results. But Suggestions need active management and investment to keep them fresh and useful.

  • Current Mood