Search Theory & White Papers.
Publicly available white papers, best practices, theories and publications about search related topics.
http://labs.google.com/papers/ Google Research Publications.
IBM Clever Project Dozens of great papers on search processing, link mapping, intelligent spidering processes.
DiscoWeb Applying link analysis to the web. DiscoWeb is what became of WebSpider and eventually the likes of Alta Vista & Teoma
Hilltop Search Engine based on expert Documents.
IR Theory book online This is a forthcoming book to be published on Information Retrieval theory.
Google This is the original Google paper by Sergey Brin and Larry Page