Vestris Alkaline, from Switzerland, has been around for a long time but is still very actively updated. Running on Unix and Windows NT, it has a web crawler than can handle multiple sites, with extensive rules options for including and excluding pages by url and extension. It is mainly focused on web pages but external filters allow indexing of XML, PDF, Microsoft Word, WordPerfect and other documents. Can handle password and Windows NTLM access control, but displays all results (no hit-level authentication). Query features include internet and Boolean operators, wildcards and number search; admins can adjust results weighting using a local GUI configuration interface. Standalone search server can run on any port. Written in C++ for binary distribution, but source code licensing is available. Low price: free for noncommercial sites, $350 for commercial sites.
The Fluid Dynamics Search Engine is a Perl CGI script that performs nicely on sites below 10,000 pages. It can crawl links or read a local file system to gather text, HTML and PDF files, and includes extensive controls for excluding pages. Search supports Internet query operators, Boolean operators and quotes. There's an option to allow public submission of URLs for topical portal search, and the admin is all done via browser interface. Only $40, runs on Unix, Windows, Mac OS X, and the documentation is excellent.