corner Search Engines and Directories, how to use them, list of the main ones and how to include a site specific engine into a site.  corner
space design

Design

 

 

corner

How to use Search Engines and Directories

corner
HTML Tutorial

    bookmark

Web Design
Page Contents

Search Engine Introduction

 

definition
4 aspects of engines

List of Search engines and Directories   Google, Google-Images, AltaVista, AOL, Ask Jeeves, dmoz, Excite, HotBot InfoSeek, Inktomi, Looksmart, Lycos, msn, Northern Lights, Yahoo, UK Plus, Vivisimo, WebCrawler, Web Page Archive

Yahoo Report dead links

Multiple Search Engine
Copernic

How to use a Search engine

Exercise 1

More search techniques

Integrating a Search engine into your site

Exercise 2 - Demo

 

Advanced Searches

Exercise 3

Exercise 4

Boolean Searches ( + - AND OR )
Wildcard Searches ( * ? )
Stemming

Further Information  

How to submit your web site to engines and
optomising your web site ranking.

 

Site Submission and Site Promotion has been moved to another page.
Site Promotions

Top of Page.

Search Engine Introduction

From my Web Terminology page the definition for search Engine is 'A search engine archives the contents of web sites into a searchable database. By entering a suitable keyword(s) or phrase into the search engine, you can obtain a list of pages that contain the entered word(s)'.

Search engines send out software robots or spiders to roam around the Web. They note the words on a page and include them into a searchable indexed database.

An alternative to a search engine as stated above is the directory, where sites are indexed into a database of searchable categories and sub categories.

As a webmaster you should be interested in 4 aspects of engines. Steps 3 and 4 are covered later in the course in Tutorial 14 Site Promotion

  1. How to use a search engine to efficiently search for information, this will mean learning some advance search techniques, which are very easy to do.
  2. Integrating a search engine into your own site.
  3. How to submit your site to engines.
    1. Then once listed, how to promote your site by getting your site ranking higher up that list.
  4. Keeping an eye on similar sites to your own.

Each engine uses different methods on the putting together of there database of web pages, not all sites are included and sites that appear in more than one search engine may have a good ranking (position in the returned results) in one search engine and a very poor ranking in others.

Therefore you should not be surprised if the results of an identical search using different engines results in vastly different pages being suggested.

Once you have mastered "How to use a Search engine" in the next section, and "Advanced Searches" in the following section. You should try a few searches using identical carefully selected keyword(s) or phrases in several of the Engines, and the sites that combine results of several Engines. These are listed below.

You should experiment with both simple and advanced searches
See which engines return

  1. The most suitable sites
  2. The highest percentage of suitable site.
  3. The clearest of site preview
  4. A speedy result.

Give a mark to each search engine, the highest marked engine, you will naturally use the most.

Top of Page.

space

 

 

space
Search this site for a word or phrase. Place phrases inside "double quotes"

Buy Domains from 1&1 and Save Money!

Buy web hosting from 1&1- Get More for Less Money

 

List of Search Engines and Directories

From a list of several hundred engines I have selected several of the top ones that should cater for the requirements of most people.

Google www.google.com This is my favorite search engine. It has over a billion pages indexed in it's database. Many pages are cached and can be retrieved, if the site closes down.
Google-Images http://www.google.com/imghp?hl=en Search for Images
AltaVista

http://www.altavista.co.uk
http://www.av.com
http://www.altavista.com

AltaVista is popular and powerful but a bit slow. The front page is a bit overcluttered.
AOL http://www.aol.co.uk
http://www.aol.com
AOL, besides having a search engine, powered by DMOZ, and Web browser is a well known ISP.
Ask Jeeves http://www.ask.co.uk Ask Jeeves is different from the others in that your query takes the form of a question. Some strange results can be obtained.
DMOZ http://www.dmoz.org The DMOZ (Directory MOnZilla) Open Directory Project, powers AOL, Netscape, Lycos and HotBot.
Excite http://www.excite.co.uk Excite has many add-ons on the home page. Owned by Tiscali
HotBot http://www.hotbot.com powered by DMOZ
InfoSeek http://infoseek.go.com  
Inktomi http://www.inktomi.com  
Looksmart http://www.looksmart.com/  
Lycos http://www.lycos.co.uk Lycos is powered by DMOZ
MSN http://search.msn.co.uk/  
Northern Lights http://www.northernlight.com Specialised financial and business searches
Yahoo http://www.yahoo.co.uk Yahoo is one of the best known directories.
UK Plus http://www.ukplus.com  
Vivisimo http://www.vivisimo.com  
WebCrawler http://www.webcrawler.com  
Web Page Archive http://www.archive.org The Waybackmachine - Very useful reference archive of cached copies of over 10 billion web pages going back to 1996.
Yahoo
Report dead links
http://www.yahoo.co.uk
http://docs.yahoo.com/docs/writeus/deadlink.html
 
SearchHippo http://www.searchhippo.com/ I have just submitted to this site 28May02
Ixquick http://www.ixquick.com/ This engine is different from the others. Giving a star rating. Also lists the position of the site in in the following engines: AOL, EntireWeb, Espotting, Go, LookSmart UK, Lycos UK, Mirago, Overture UK, UKPlus, WISEnut

 

Top of Page.

Multiple Search Engine

There are a few programs available that you can obtain from magazine cover disks, or download from the web, that you instal and run from your hard drive. These will prompt you for your search query, check out several large engines and then provide you with the combined results.

Copernic

Copernic is regarded as the turbo charged web searching tool.

Downloaded from XXXXXX___________________________________XX

  • It queries multiple engines.
  • Customise options.
    • Validation of results to remove dead links.
    • Auto downloading of result pages, if required.
    • Skins
  • A very good search tool.
  • Free.
  • New users should look at the legend window ( From 'Window' 'Legend') to make sence of the 'Symols' against each result.

How to use a Search engine

In order to search for some information you enter into the input box of the search engine one or more words or phrases. Phrases are enclosed in "double quote marks". These works are called keywords

Exercise 1.

Step 1 If you are not on line, go on line now. Normally you would open what I believe is the best search engine available www.google.com, but that is not required because for this exercise I have brought the search engine to you, see below.

Step 2 Assumption.

Assume that you are studying the Open University course M206 Computing an Object - oriented approach, and wish to find a tutorial on the Smalltalk programming that is relevent to the M206 course one of the subjects on this course.

All the words in bold text in the above sentence are possible suitable words or phrases to search for. In Jan 2002 I searched for some of the above and the number of returned matches are shown in the table below.

Table of results on a single word or phrase
Keyword Returned matches
"Open University" 300,000
M206 6,310
"Object - oriented" 1,200,000
"Computing an Object - oriented approach" 341
tutorial 4,480,000
Smalltalk 518,000
programming 13,700,000
"Smalltalk programming" 3,010

Top of Page.

Step 3 The secret of successful searches is the careful selection and combination of your keywords. From what I have said in the assumption
Assume that you are studying the Open University course M206 Computing an Object - oriented approach, and wish to find a tutorial on the Smalltalk programming that is relevent to the M206 course one of the subjects on this course.

Pick out the 2 main keywords

Obviously Smalltalk is the main one, and because the assumption said, relevent to the M206 course, I would say M206 the other, tutorial being a strong contender.

Step 4 For the rest of this exercise you are going to use the following search engine that I have installed below

 

Google
Search WWW Search tutorials4u.com

Ensure that the Search WWW option is selectet

Enter the following words into the input box.
Smalltalk M206
then click the Google Search button.

This search engine will look through its database for pages that have BOTH these words on the page.
Note most other engines will present pages that have EITHER of the 2 words on a page.
To find pages that have BOTH these words in them, these other engines would require you to enter either of the following
+Smalltalk +M206
Smalltalk AND M206

Step 4 Check the results

You should look at the result page produced. Look for the number of pages found near the top of the list, it was 457 when I tried it.

Note how the search words you typed in are displayed in bold within the extracts from different sections of the page.

Read a few of the entries and see how near they are to our requirements.

Look at the top, or near the top of the list for my M206 site with an entry similar to the following.

Smalltalk programming tutorial, and information for students ...
... Smalltalk Tutorial, M206 and the OU. ... Master Index for the OU Course
An index for the M206 OU course and Smalltalk methods used in the tutorials. ...
members.aol.com/M206ou/m206/ - 23k -

Clicking on the either of the 2 links in the Google entry, or in the above paragraph will take you to the Smalltalk and M206 site

Webmaster tip:
Just under the entry look for the word cashed, and click this. This is the page as cashed by Google the last time you they sent there robots visiting the site. If you are frequently updating your site and place a date when the site was last updated on your homepage then you will obtain an indication of when your site was last visited by the robot and the site database uploaded. This does not work for many of my sites because the date is generated by a JavaScript and the cashed process will update the date to todays date.
You may find this useful when you get your site listed on the Search Engines.

Top of Page.

Exercise 2.

 

Step 1 Add the word tutorial to the other 2 words and try again

Step 2 Take a note how the search results have now been cut down in number. By the careful selection of your search criteria you should be able to obtain the best possible sites for your purpose.

Top of Page.

space
space

 

Searching for a Phrase

To search for a phrase enclose the phrase in double quote marks. E.g. "web design"



Students can save money by buying Student Licence Software.

Details

 

Integrating a Search engine into your site

Check out these sites for free engines that you can integrate into your site.

www.atomz.com
I use this on my Smalltalk tutorial site.

Exercise 3.

Click here to see an integrated engine in action. The page will open in a separate browser window. Search for printString

I have not had time to experiment with the customising features that are offered. Would like to hear from any body that has.

www.searchengine.com
Have several free engines that can be integrated into your site.

Top of Page.

Advanced Searches

Many engines have an Advanced search page, where you can enter additional query details that should improve your chances of returning relevent sites.

Boolean Searches ( + - AND OR NEAR)

Phrases

Phrases should always be placed within a pair of quotation marks. E.G.
"turbocharged engine"

The engine will look for an exact database entry for the phrase, finding turbocharged engine, but not finding turbocharged car engine
The useful search query to use here is
turbocharged NEAR engine
which looks for the 2 words being in near proximity.

If you enter a search query
T171 HTML tutorial
into the Google search engine you will obtain results that contain ALL three of the keywords. Entering the same into most other engines and you will obtain results that contain ANY of the 3 keywords. In these engines you would need to use Boolean search techniques to obtain sites that contain ALL the keywords as follows:
T171 AND HTML AND tutorial
or you could use the following alternative method
+T171 +HTML +tutorial

Many words have alternative meanings or can have abbreviations and you may wish to accept either of the alternatives, then the boolean word to use is OR, e.g.
+T171 +HTML AND tutorial OR lesson OR Instruction
"Member of Parliment" OR MP

Top of Page.

Exercise 3. Using + AND OR

Use the above examples in both Google and other engines.

As you use the different engines, notice which ones you prefer. You should do this each time you use an engine, with the aim of finding the one that suits you. The following may help you in making this decision.

  • How good are the top ranking results in answering your search criteria.
  • Is the short intro to the site helpful.
  • Does the engine highlight the keyword(s) or Keyphrase(s)
  • Does the engine supply a lot of unhelpful "paid for sponsership" sites at the top of the list.

Top of Page.

Exercise 4. Using - NOT

These boolean expressions are used to reduce the number of results that you obtain, by getting rid of many of the unwanted sites. For this exercise you are going to try and find all of the sites that I have written under the name of John McGuinn, see table below. Your search will also bring up some other sites that link to my sites and have used my name.

I have used my name for this exercise because it is not very common, it also is not a high ranking keyword on my sites. This should therefore give you a good insite into this very useful technique.

The 3 sites of mine are:

Site Some of the Main Keywords
http://www.tutorials4u.com/html/
or http://www.tutorials4u.com/
T171, T170, HTML, tutorial, Web design
http://members.aol.com/M206ou/m206/ M206, Smalltalk, tutorial
http://members.aol.com/freetutorials/c/ T223, C, tutorial

There are more John McGuinn's in the world, and it is your task to remove these from the search results by using the _ or NOT boolean expressions.

Step 1. For this exercise use google. Search for "John McGuinn" and note the number of results.

The key to reducing the number of results is to try and spot keywords in the other John McGuinn's sites that are none releventto my sites as given in the table above. I do not want to give too many clues to doing this exercise. But let's assume that entertainer crops up in a lot of the results, this is certainly none relevent to my sites, but you should look for a better keyword or keywords to use.

Step 2. Search for
"John McGuinn" -entertainer

Top of Page.

Wildcard Searches( * ? )

Stemming

Stemming is a process of finding additional words with different endings. E.G. If you typed in help as your search query stemming would find help, helping, helped, helper etc. Some engines use stemming automatically, but the same result can be achieved by the use of wildcards in the non stemming engines.

Wilcards can be used to include the:

  • Word plurals
  • Stemmed words
  • Words you are not sure of the spelling.

The wildcard characters are * and ? and these normally stand for a single or group of characters, and a single character respectedly. Not all search engins work the same way and some engines reverse these uses. A quick look in the engines help section or a quick search experiment should verify the swituation.

Exercise 5.

Try the following and note what words are brought up for the keywords.

+T17? +HTM? +tut*
+T17* +HTM* +tut?

 

Top of Page.

How to Submit your Web Page or Web Site to Search engines.

Web Site Submission and Web Site Promotion has been moved to another page.
Site Promotions

Further Information

The Open University T171 web site has Search Engine information. These pages may have restricted access unless you are a T170 or T171 student

 

corner
Design
Previous Top of Page Next
Design
corner
space

 



In Association with Amazon.co.uk
In Association with Amazon.co.uk

In Association with Amazon.co.uk

 

 

{Mid2Left}

Leisure Time - book your holiday or flights with our UK sponsors


Last minute

Save 10%
Book a Thomas Cook holiday Book a Thomson holiday Book a Portland Holidays Direct
Thomas Cook flights Thomson holiday flights Flybe Cheap flights British Airways flights My Travel flights
The London Pass - Adult The London Pass - Adult

Welcome to The London Pass, the brand new leisure pass that lets you experience the very best that London has to offer. With free entry to over 50 attractions, free public transport and the ability to beat the queues, The London pass helps you to make the most of your stay.


More Tutorials by John McGuinn

M206
Home page of a tutorial in programming in Smalltalk, a object-oriented programming language. This is an ideal tutorial for anybody learning Smalltalk and of particular interest to students on courses: M206 at the OU Open University, and course CSC517 at NCSU North Carolina State University

C programming tutorial
Home page of a tutorial in programming in C This is an ideal tutorial for anybody learning C programming language, and of particular interest to students on courses: T223 at the OU Open University.

tutorials4u.com
Home page of tutorials4u.com

Leeds my home town

Relax in the sun.

Benidorm   Tenerife   San Marino apartments to rent

 

Top

AireWeb Web Design
Leeds Web design

Copyright © John McGuinn 2001-03

.