SearchTools Blog

Fundamentals of Enterprise Search workshop slides

Tue, 17 Nov 2009 01:31:32 GMT

the workshop went really well, interesting discussions and questions.

(Read more ...)

Enterprise Search Summit West: Nov. 16-19, 2009

Sat, 14 Nov 2009 00:46:07 GMT

The Enterprise Search Summit (west and east) are always great meetings, and very productive for me. The presentations are less vendor-brainwashing and more valuable insights and case studies. I learn a lot from those exhibitors who have programmers and product managers for me to talk with, the group lunches, and the hallway conversations.

(Read more ...)

Apache Solr 1. and Apachecon Meetup notes

Fri, 13 Nov 2009 23:50:52 GMT

Solr is a leading open-source enterprise search package: this new version is is built on Apache 2.9.1 and Java 1.5 VM.

(Read more ...)

Looking at real-time and near-real-time search

Tue, 03 Nov 2009 03:20:46 GMT

Real-time vs. near-real-time search

(Read more ...)

open source search levels the playing field

Fri, 09 Oct 2009 02:30:56 GMT

I wrote a NewsBreak for infotoday.com about the new version of the open source search engine library Lucene 2.9 and associated projects. Stepping back a bit to look at the whole thing rather than a feature summary. And I ended up with an even deep appreciation of open source search engines in general and in particular, the Lucene family of search-related tools (language ports, Solr search engine, Nutch web crawler, Tika file format converter, and more). They are as capable and powerful as many commercial enterprise search engines.

(Read more ...)

Lucene 2.9 to be released very soon!

Thu, 24 Sep 2009 19:07:10 GMT

Apache Lucene is the most prominent open source search engine, and powers search on a lot of really interesting sites. The new version, 2.9, has internal improvements, re-factoring and new functionality.

(Read more ...)

good introduction to search analytics article

Thu, 24 Sep 2009 00:23:32 GMT

Very clear and well-written: Internal Site Search Analysis: Simple, Effective, Life Altering!. It covers both search on a site, and search from search engines leading to a site, with very useful examples and screenshots from Google Analytics.

Enterprise Search Summit & Infonortics Search Engines Meeting news

Thu, 03 Sep 2009 20:36:22 GMT

Information Today, Inc (producer of the Enterprise Search Summit), has announced that it's acquiring the Infonortics Search Engines Meeting.

(Read more ...)

Google CSE - search and results interfaces

Fri, 24 Jul 2009 22:36:30 GMT

There are four kinds of Google Custom Search Engine search / results interface:

Simple form, showing results on a normal Google-hosted page with minimal customization (example)

Form with links to a template page, JavaScript inserts iframe with search results pre-formatted (iframe example). Fits into site colors, design and navigation, but has minimal other results customization, including the width of the results list box.

Custom Search Element - AJAX object draws a search form, JavaScript can draw result list anywhere (AJAX example). This is the new cool programmatic toy, and with CSS, it's very flexible for customizing width, colors, sizes and styles and more.

XML query and result protocol (paid Site Search only) is by far the most comprehensive and flexible. See the XML Protocol Reference for extensive documentation.

If you have any questions, suggestions or corrections, please comment on this post (non-account comments allowed but screened), send email to nets 9 at searchtools.com, or contact me through the site contact form.

Netflix Recommender Prize Won (probably)

Thu, 09 Jul 2009 00:14:52 GMT

Netflix has posted a contest for improving its own movie recommendation system by at least 10% and the prize is a million dollars. People have been working on this since 2006, and there have been several Progress prizes. Finally, four teams merged to create the BellKor's Pragmatic Chaos Team, which added temporal dynamics to the recommendation weights. This beat the current Netflix recommendation algorithm by 10.0%%.

(Read more ...)

thoughts on search engine comparisons

Wed, 08 Jul 2009 00:09:32 GMT

Vik Singh wrote an in-depth post about his comparison of open-source search engines. He tested the default configurations for Lucene, zettair, sphinx, and Xapian, with a nod to sqlite.

(Read more ...)

Recommended Book: Search User Interfaces

Fri, 26 Jun 2009 03:46:17 GMT

Marti Hearst, Search User Interfaces, 1st ed. Cambridge University Press, Septermber 2009. [Online: www.searchuserinterfaces.com]

(Read more ...)

Google CSE AJAX API - Using CSS on the search results

Thu, 25 Jun 2009 00:20:56 GMT

CSS styling of AJAX search results

For customizing the fonts, colors, size, and styles of the Google Custom Search Engine (CSE), there is a many-layered hierarchy of div class names. For example, I found the right name to to turn all the result item titles red (even the bold subsections). I had to use the name of my search results for the browser to render correctly:

(Read more ...)

Decoding the new Google Custom Search API

Tue, 23 Jun 2009 03:50:55 GMT

Google has released a new version of their Custom/Site Search service, and added an "Element" -- a wizard-driven JavaScript that non-technical users can copy and paste to their web sites, even blogs which do not allow uploading. Search Tools has a new Analysis of the CSE and AJAX API. I also wrote a fully-commented sample code with a live version on the same page, because this is much harder for non-programmers to customize than the forms or even the Site Search XML interface (paid version only). I'll be doing more on customizing and functionality and display during this week.

(Read more ...)

Lucene/Solr meetup notes

Fri, 05 Jun 2009 20:50:17 GMT

Lucene/Solr Meetup, June 2009

Notes by Avi Rappoport, Search Tools Consulting

Solr 1.4, Near-real-time indexing, Payload efficiency, Trierange, Query parser framework, Zevents, Xoopit, Lucid search, Stopwords are obsolete, OpenRelevance

Meetup event info

(Read more ...)

Best Practices for thumbnails in search results - article

Thu, 28 May 2009 23:32:39 GMT

Making $10,000 a Pixel: Optimizing Thumbnail Images in Search Results :: UXmatters

(Read more ...)

Twitter Search Has Big Ambitions

Thu, 28 May 2009 22:40:19 GMT

My overview of the state of Twitter Search on infotoday.com - it doesn't even try to do relevance ranking right now, so it's not exactly a Google killer, despite the hype.

#AmazonFail: garbage in, garbage out

Tue, 21 Apr 2009 01:53:05 GMT

My article on #amazonfail is up at Amazonfail: How Metadata and Sex Broke the Amazon Book Search:

Amazon failed in a big way on Easter weekend. As the largest bookstore in the world, if a book does not appear in its lists or its search results, the book practically disappears. The event now known as #AmazonFail involves a great cast of characters-books, metadata, sex, search results, traditionally disenfranchised groups, a possible hacker, the Kindle, the absence of institutional response, and the emergence of Twitter for sharing information very quickly on a massive scale.

So that's what I was working on last week.

Enterprise Search Summit / NY, May 12 - 13 2009

Wed, 15 Apr 2009 21:10:43 GMT

The Enterprise Search Summit early registration deadline (save $100) is this Friday

(Read more ...)

Two new Search / Information Retrieval textbooks

Thu, 02 Apr 2009 22:42:38 GMT

Introduction to Information Retrieval by: Christopher D Manning, Prabhakar Raghavan, Hinrich Schütze; July 2008 from Cambridge University Press [disclosure: the link has my amazon affiliate code]

(Read more ...)

Openfind Enterprise Search (OES) - New SearchTools Report

Wed, 25 Mar 2009 22:04:21 GMT

March 25, 2009

Openfind Enterprise Search (OES)

Openfind is a leading enterprise search engine company in Taiwan, providing search to many government departments and corporations since 1998, scalable to over 50 million items in their standard licence. In addition to documents, it can index text and some numeric content from relational databases, off-loading the search and spreading the server load.
(Read more ...)

searchtools links

Tue, 10 Mar 2009 23:31:02 GMT

Interesting stuff I found today:

(Read more ...)

some thoughts on Wolfram Alpha search

Mon, 09 Mar 2009 21:24:46 GMT

Stephen Wolfram, the guy who made Mathematica into a big software company and then wrote a book, is now plugging his new Wolfram | Alpha search engine, meant to compute knowledge (or at least answers) instead of return lists of pages which may have answers. This is not new, TREC has done it for decades.

(Read more ...)

More about file format parsing tools for indexing

Fri, 06 Mar 2009 01:53:15 GMT

This is a follow-up to my article about file format access tools and Lucene Tika.

(Read more ...)

Tika: open source access to text in many formats

Fri, 27 Feb 2009 23:05:11 GMT

Search engines need text to index: this may seem obvious but the devil is in the details. Extracting text is easy when working with txt, html or xml files, but much more difficult for binary files, including MS Office and archive formats. So search indexers need to use file format parsers, also called "filters". These can access the binary file formats, extracting the text and keeping track of whatever structure is there. Some file parsers are better than others, and all of them may need updating: as Microsoft switched from the proprietary format to their xml (doc -> docx), the search indexers need updated filters to read the new formats.

(Read more ...)