WebCrawler Search Engine Optimization Strategy

WebCrawler search was created Jan 27th, 1994, by University of Washington student Brian Pinkerton. Originally a desk top application search tool, WebCrawler search went live on the web April 20, 1994. WebCrawler joined AOL in 1995, but was later acquired by Excite in 1997, and in 2001 it was bought by InfoSpace.

WebCrawler is a meta engine. Instead of sending out a spider to crawl various web sites to create its index, WebCrawler spiders other well-known SE’s and directories. Since it builds its index this way, search engine optimization by an SEO service is an important step to making sure your site is found and listed in these databases. The SE’s and directories it crawls include: Google, Yahoo!, MSN Search, Ask Jeeves, About and Looksmart.

WebCrawler’s spider, now called Hunter (formerly Spidey), looks for the most direct terms and phrases from the engines and directories. WebCrawler’s automatic categorization technology helps refine results and automatically organizes them into categories based on words and phrases contained in the results. Returned results also show where they were taken from. WebCrawler’s relevance algorithm gives more weight to titles than to body text.

WebCrawler search uses the “IntelliFind” algorithm, which is also utilized by InfoSpace’s other meta engines. IntelliFind compares user keywords to data from a number of sites. It “predicts what kind of information you are looking for based on the overall popularity of the keyword(s). The engine then tries to find matches with things like movie titles, musician names or seasonal items, then recommends matching content.”

WebCrawler search displays a ‘Recent Searches’ feature that shows past items searched for. Items under ‘Recent Searches’ can be removed, and this feature can be hidden at any time. Radio buttons are also available to help find searches within specific categories, and there is an advanced search feature to increase the accuracy of a search. Results are provided under: Web, Images, Audio, Video, News, Yellow Pages and White Pages.