ALIA home | Online home | search | papers | keynotes | program | sponsors | satellite events
|
Andrew Osborne
Is your website search engine friendly as well as user friendly? Search Engines and Directories are like the yellow pages of the Internet. Statistics show that, depending on your industry, up to 82 per cent of potential customers will try to use a search engine or directory to try to find your website. Common website design practices make many websites invisible to search engines or effectively hide the content. This paper looks at how to avoid the ten traps of website design that make most websites extremely difficult for potential users to find and what are the questions you should be asking your web designer.
Many organisations spend large amounts of time and money to build websites. If these websites cannot be found by the potential users/customers then this is wasted money. This paper will examine key issues such as:
The purpose of a website varies from communication internally and externally to advertising and supplying goods and services. Not only is the money wasted developing the website if it cannot be found by the right people but if the purpose is not achieved as a result of not being found then in some cases the consequences can be dire.
Unless companies have massive offline (none internet) advertising budgets and can afford to plaster the web address in front of peoples faces by traditional advertising methods such as Bill boards, newspaper, sides of buses, backs of taxies and television exposure then the most effective way that many of the users will find their way to the website is via the search engines or a directory. Search Engines and Directories are like the Yellow Pages of the Internet.
Statistics [1] show that, depending on your industry, 56 per cent to 82 per cent of potential customers will try to use a search engine or directory to try to find your Website. If you are not listed in the major search engines or directories you missing out on potential customers/visitors and they may find your competition instead of you.
Not listing a website in the major search engines and directories is like building a traditional bricks and mortar business beside a busy road, surrounded by other businesses and not putting up any signs or doing any advertising. There are other ways people will find a website, such as word of mouth and following links from other sites but the most effective way for most users when the address is not known is using the Search Engines or Directories.
To understand the problem it is necessary to examine how search engines work. Usually the Webmaster or the web design team of the website will submit the site address to relevant Search Engines and Directories with appropriate information that describes the site. The site will then go onto a list to be visited by a specialized software applications called Search Engine 'Robots' or 'Spiders' as they are often called in the World Wide Web environment. In the case of a Directory (e.g. Yahoo or DMOZ) it will be a human editor that visits the site. The Search Engine Spider or human editor then evaluates the site according to set criteria, extracting information such as title, keywords, themes, and a description. This information is then added to a database which a searcher can query through the Search Engine Website.
What are the Search Engines looking for? Very simply they are looking for visible relevant content. This allows searchers to find the website based on the content of the website not just the name of the company. Thus it may be possible to find the website based on a product or service for example offered in a specific geographic region.
There are a number of things that can prevent the Search Engine Robot 'reading' a website and extracting this information from the content. These include content being contained within:
Also the website may have been banned for any one of many reasons.
The problem of content with in graphics arises from the process the search engine robot uses to 'read' the website. It looks at the program code that makes up a website e.g. the Hypertext Markup Language (HTML). Individual words are identified by pattern matching the letters or characters that comprise those words against the Search Engine Dictionary. If the word is contained with in a graphic it is impossible to extract individual characters and it is effectively invisible. Search Engines cannot understand or read graphics.
'Flash' and 'Shockwave' are applications that can be used to dramatically increase the impact the website has on the viewer. These applications include a number of special effects that are stunning. Unfortunately Flash and Shockwave files along with sound files and PDF files cannot be read or understood by Search Engine Robots for the same reason as graphic files. Thus information or content contained within these are lost to Search Engines and users will not be able to locate the site by searching for this content.
Frames are often used to help layout a page to assist the user to easily navigate through the site. This involves 'Framing ' of a page within panels that can change independently of each other or stay constant through out the site. Frames have a negative side in that they have only limited support by most Internet Search Engine Spiders. Most Spiders don't read frames. Instead they read the body of the frame-setting document only and the content of the 'Noframes' tag. Most webmasters, simply place a statement like 'Your browser does not support Frames.' inside that tag, which does nothing to improve a site's visibility.
A quick search of the at Google Search Engine (google.com) for the words 'your browser does not support frames' indicates that Google currently does not read frames but will index the contents of the Noframes tag. That's the content you'll see displayed on the Google results page. Searching Google for listings for the following: 'The Option Strategist - Technical Support'.
Google displays:
The Option Strategist - Technical Support
... If so, your browser does not support frames and you must install a new browser
to enjoy the benefits of this widely used and accepted website feature. ...
optionstrategist.com/aboutus/support/ - 26k - Cached - Similar pages
As Search Engine Technology progresses, frames and some of the different file formats such as PDF will become less of a problem.
Dynamic Pages are pages constructed 'on the fly' from information held in a database. The URL usually has a '?' in it which stops Search Engines from spidering these pages.
For example in the following address: http://www.bookshop.co.uk/ser/serdsp.asp?shop=1andisbn=0345407970andDB=224 a search engine robot will not be able to read anything after the '?'.
Tables are often used to layout a page.
Search Engine spiders read tables differently from human users. This is because humans read tables going from cell to cell across the page, left to right, in rows while search engine spiders read in columns down the page.
Poorly laid out tables can push relevant text further down the page for a Search Engine robot making it appear less relevant or not important at all.
There are a number of ways that a website can become banned from a search Engine and thus not listed. Searchers will not even be able to locate them by the name of the business. It will be as if they don't exist.
There are those things that are done innocently that Search Engines now do not like. For example:
Then there is the submitting of pages designed to try and mislead a search engine. For example:
This sort of attempt to mislead the Search Engines is also know as Spaming but it is not to be confused with e-mail Spaming. It is done in the hope of achieving a higher ranking on search results (in other words appearing towards the top of the results of a query). Often this is done by web designers inexperienced with Search Engines or aggressive Search Engine Optimizers (SEOs). The higher the website ranks in the results of a query, the more likely it will be reviewed by a competitor to see how it achieved its ranking.
Spam is not very difficult to spot.
If a website is outranking someone in a competitive market through use of a spamming technique, the odds of it being reported and penalised are high.
These are just a few things out of a growing list that will result in a website being banned from Search Engines. It is very difficult and sometimes impossible to reverse the search engines decision to ban a site. In some cases it is easier to relocate the site to a new domain name.
What is the Answer? Can you keep the fancy visually appealing site?
Can you still use these fancy web page improvements?
Yes, you can still use them, but you must remember that Search Engines robots/spiders need visible relevant content and take appropriate steps to achieve this.
Some things that will help with visible relevant content for Search Engines:
On Intranets (internal networks) good metadata is very important to resource discovery if the internal Intranet Search Engine supports it. Metadata can greatly enhance the discoverability of a relevant page or sub site on large intranets especially. For example it allow the introduction of synonyms in the keywords thus greatly improving the chance a user will find the page even if the search term they used does not exactly match the term used in the content of the page.
On the Internet metadata is not so useful as it once was or should be. Unfortunately too many aggressive Search Engine Optimizers whose job it is to design websites to rank very well on search queries, have abused the metadata tags on websites to the extent that the Search Engines no longer trust the information contained with in them.
Websites can be checked for possible problems through Search Engine robot/spider simulators which show you the website as they see it. Two of these can be found at the following web addresses:
http://www.searchengineworld.com/cgi-bin/sim_spider.cgi
http://www.delorie.com/web/ses.cgi
Lynx is a text-mode web browser located at http://www.delorie.com/web/lynxview.html This service allows web authors to see what their pages will look like (sort of) when viewed with a text-mode web browser. This in many ways will help simulate what a Search Engine Spider sees.
When submitting sites to Search Engines and Directories, it is not necessary to submit to hundreds. Submitting to ten key Search Engines and Directories will give you close to 80 per cent coverage if chosen well. This is because different Engines and Directories share information between each other. One example of where to promote or conversely check to see if your site is listed is in the following list of Search Engines and Directories:
The following are questions you should be asking your web designer.
Managers need to be aware that websites are generally designed to be user friendly but they also have to be Search Engine friendly as well.
If a site is invisible to Search Engines it probably poses problems for people with disabilities such as the visually impaired for example. This can be discrimination and is dealt with by the following
Australian legislation:
The good news is that many of the techniques of website design that improve accessibility to websites for people with disabilities also improve access to the content, helping to make it visible for Search Engine Robots.
Information in the area of Search Engines changes frequently so for up-to-date information on Search Engines and Directories the following sites are excellent:
http://searchenginewatch.com
http://spiderfood.net
The answer to the question 'You have a great website so why cant people find it?' comes down to three words. Does the site have visible relevant content?
[1] Driving Customers, Not Just Site Traffic
Forrester, 28 March 2001 (Data from October 2000) at http://www.forrester.com/ER/Research/Brief/0,1317,12053,FF.html
April 2000, RealNames Survey reported in Search Engine Watch , 2 June 2000 at http://www.searchenginewatch.com/sereport/00/06-realnames.html