AWARE Competitive Intelligence Resources
Recommended Web Sites for CI Research

Sites checked: 23 August 2010

Topic Index
Global / Major Search Engines
Search Directories, Smaller Search Engines & Speciality Search Engines
Blog, news group, shared bookmark and discussion forum searching
Invisible Web Searching

Global / Major Search Engines

The following are the main global search engines. There are also a number of other older search engines still available that use the databases of the search engines below. For example, Go, Inforseek, Altavista and Alltheweb all use the Yahoo! search functions and database, etc.

• Google - Currently the most popular search engine - with considerable flexibility for advanced searching, including date, boolean options, etc. We recommend that you check the Google advanced search help page to learn the full capabilities of this search engine.
• Google UK - When searching for local country information it is generally easier and better to use the local versions of Google (and other search engines) as these help narrow the search down to the country concerned. So - for France, search on Google France; Germany: Google Germany; China: Google China, and so on (although for some countries there may be problems reading the language - as there are multi-language versions of Google as well as multi-country versions).
• As well as the regular Google, there are a number of specialist versions allowing you to search academic journals (Google Scholar), Recent news (Google News) and archived news (Google News Archive) as well as the content of numerous books (Google Books). Google frequently enhances what they offer and it is worth checking out Google's full options list.
• Yahoo! - Like Google, a major search engine with a variety of search options - including some not currently available on Google (for example the Linkdomain field operator allowing searches for all sites linking to a web-site). Also check out the local country versions (e.g. Yahoo.co.uk) and other Yahoo options, including the Yahoo site directory. Yahoo now uses Microsoft's Bing search engine for results, and so is less useful than in the past.
• Bing - Bing was previously known as Live and MSN Search from Microsoft. These have been revamped and has changed into Bing. As with the other search engines, there are a number of advanced search options - and Bing has several unique features making it worth considering as a search tool.
• Ask - also known as AskJeeves - is another major search engine - offering a different patented approach to finding and ranking sites using technology developed by Teoma which Ask has replaced.
• Exalead - smaller than some other search engines - offers several unique advanced search features that make it well worth using. These include the NEAR boolean operator, wild-card searching, word stemming and other professional search options.
• Gigablast - another small search engine worth trying.

Search Directories, Smaller Search engines & Meta-Search engines

As well as the major search engines described above there are many smaller, regional and specialist search services, including some industry specific services (e.g. for searching specific topic or industry areas - law, finance, etc.). Some of these are listed under the relevant sections. The search tools listed here are more general or have features that make them noteworthy.

• Browsys - a useful way of searching several search tools at once. Enter your search into the search box and then select your search tool. You can then repeat the search with several search tools and tool options - including the major search tools above, as well as their subsets (e.g. Google Scholar, Google Blog Search, Yahoo! PDF search and so on). This is a really useful tool and my major regret is that it seems to override search preferences set up with each individual search engine.
• Zuula - a similar (but less sophisticated) option to CrossEngine. However this also has some nice features not shared by Browsys - for example a Job search facility.
• Open Directory - a human edited list of millions or web-sites, where each is vetted and approved as worthy of inclusion. The Open Directory powers Google and a number of other web-site directories (although not Yahoo! which operates its own directory).
• Factbites - a topic search tool that helps give an overview of the search term, with bitesize extracts from the sites listed. Another more sophisticated fact search tool is Wolfram Alpha.
• Snap - allowing you to see the pages listed next to the search listings.
• Dogpile - one of the main metasearch tools
• Yippy - owned by Vivisimo and formerly known as Clusty, Yippy clusters search results by topic. It is a great tool for searching when you are not sure of the exact terms to put in, as its clustering approach allows you to refine your searching. It also has a great meta-search tool for blogs.
• Request.net - a small directory but focused on only a few key sites per topic, this makes a good quick reference source
• Lexxe - Natual language searching
• Accoona - Web-search tool using artificial intelligence approaches and focusing on company listings

Blog, news group, shared bookmark, and discussion forum searching

Searching blogs, podcasts, discussion forums and news groups can provide crucial information for competitive intelligence. This gives a small selection of tools that can help in this area.

• Yippy - Yippy's blog search tool.
• Technorati - one of the main blog search tools
• IceRocket - a major blog search tool
• Podscope - search Podcasts. Podscope seems to translate the audio record of the Podcast in the words, allowing searches of the content of Podcasts.
• Tile Net - a database of mailing lists and ezines. (Also consider Yahoo! groups and Google Groups - both of which have similar material although linked to the respective Yahoo! or Google sites. Google Groups has also incorporated news group archives going back over 20 years).
• Omgli - for searching web-forum discussions
• Delicious - A bookmark site which uses tags to index sites, with each site added by another searcher. Such bookmark sites offer a different way of finding information as it goes back to the early days of the web where people recommended the top sites to each other. Just visiting the first page and seeing the current "hotlist" is interesting in itself as the collection of sites is always interesting.
• BlinkList - Another bookmark site - "Real people with shared interests list the websites they care about" resulting in some excellent finds that may not have appeared in conventional searches, especially as many seem to be articles and individual pages with interesting content, rather than complete web-sites.

Invisible Web Searching and other search utilities

Research has shown that the vast majority of web-sites are never indexed by search engines. Many web-pages are now created dynamically from databases and so cannot be found. One approach is to use an invisible/deep web search tool to locate content held in subject or topic databases. The list below gives major invisible web search tools, as well as some tools to learn more about finding information and some speciality search tools for specific types of content.

• Scirus - Scirus is a science-specific search engine and searches over 300 million science-specific Web pages. Consider it for any science based search.
• CompletePlanet - search thousands of databases through CompletePlanet
• IncyWincy - describes itself as the Invisible Web search engine
• BRINT - a business database search tool (BRINT = a Business Researchers Interests)
• Beaucoup - a directory of subject specific and specialist search engines
• GeniusFind - another subject specific search engine directory
• About.com - a series of sites focusing multiple topics and each led by a subject guide who knows where things are. The sites generally link to many sources that can only be found by those who know the topic inside out.
• Wikipedia - Not strictly a deep web search tool, but still useful as a way of gaining familiarity with a large number of topics. Wikipedia also includes mini-profiles on many major organizations allowing for a quick view of the company, with more information than can be found on some other sites. Also important are external links to other sites of relevance to the topics. However Wikipedia has also come under justifiable criticism over the quality of some of its articles, which are edited by people are interested in the topics but who are not always experts or even correct. (We have come across several Wikipedia items that were wrong or inaccurate, for example)
• ArchiveGrid - a database aimed at helping historical search, giving access to lists of holdings held in a large number of libraries, museums, and other archive sources.
• The WayBack Machine - the Internet Archive at Archive.org allows you to see what sites looked like over a year ago. The site is invaluable for tracing how web-sites have changed as well as finding web-sites that are no longer active or accessible, but that once were.
• Spyfu - this is an enhancement to the also useful, Googspy. Both give information on keywords used in online advertising and similar meta-data linked to searching
• Delicious (Del.icio.us) - A social bookmarking site, giving sites that other users feel worth bookmarking and making public
• Squidoo - a directory of recommended sites, compiled by subject experts. (This differs from the open directory in that only sites that are felt to be relevant for further knowledge are supposed to be included. The lists will be much smaller but may be worth checking for specialist topics)
• Rollyo - a cooperative site where indivduals list favourite sites for particular topics. Rollyo can be useful for identifying Blog sites relevant to particular topics
  
  
• SearchEngineWatch - a great site for keeping up-to-date on what is happening in the search engine world
• SearchEngineShowdown - describes differences between the search engines, and more
• Pandia Search Central - a news resource on what is happening on search engines - aimed primarily at search engine optimisers and as a result providing a lot of information on how search engines are changing.

Our services in competitive intelligence research, competitor analysis and CI training will help you integrate and use competitive and marketing intelligence in your business, strategic and marketing planning processes. Whether you need research, advice or training, our mission is to support our clients so that they achieve their growth objectives.

For the best UK & European competitive intelligence and competitor analysis services, contact us today.

About Us Services Resources Publications Help Contact us Site Map

Copyright © AWARE 1995-2010 Last page / site update: Wednesday, June 9, 2010