All major search engines combine a manual review process with their automated relevancy algorithms to help catch search spam and train their relevancy algorithms. Abnormal usage data or link growth patterns may also flag sites for manual review.
- Inktomi Spam Database Left Open to Public – article about Inktomi’s spam database from 2001
- Search Bistro – links to Google’s General Guidelines on Random Query Evaluation [PDF] and Google’s Spam Guide for Raters
- Google Image Labeler – example of how humans can be used to review content