How the Search Engine Spiders Work
Search engines have to examine the websites that people submit to them. Though most search engines are automated, some search engines like Yahoo!™ employ people to check each website registered with them. The automated search engines use specialized software called "spiders" or "robots" to do this. Spiders help search engines deliver accurate search results by determining the relevance of the phrases and keywords a web surfer uses to the website. Spiders “crawl” through websites, analyzing text content and follow hyperlinks. The information it collects about the website is used to determine how it should be categorized and ranked. The way the spiders function are so critical, that their functionality and the way they operate is a closely guarded secret. It is critical however to understand as much as we can about them, and to use that knowledge in the design phase.
The most important thing to keep in mind when designing your website is to see your site from a spider's point of view. A spider can only analyze text and words that are in a structured format. That is why it is so critical that the proper software is utilized in construction that structures the website in the proper way. For example: A site designed using frames rarely ranks well on a search engine. The coding to develop a frames platform does not have a conventional format-- The code content of a frames site creates content that is jumbled about the page in different code sections and script excerpts which confuses the spiders.
Also, a spider needs to know immediately, what it should look for when it crawls your site. Utilizing meta keywords is the best method for doing that. Otherwise, spiders will try to guess the content on your page and won't necessarily be successful -- getting ranked high for something unrelated to your site isn't helpful at all.
Descriptive and targeted meta keywords aren't the only thing search engine spiders look for. When you use meta keywords, a spider tries to find out how relevant those keywords are for your site. If that keyword or derivative occurs on the page multiple times, the spider will see your site being more relevant to those particular words than words which only appear once.
In addition, some spiders look for the position of a keyword. If it locates a keyword in the page title, or in the first six lines of the body, it considers that to be very significant. The "weight" of a keyword is a big factor, as well. If a keyword appears three times in a page with one thousand words, that keyword has a lower weight then if it was on a page with thirty words. Pages with heavily weighted keywords are considered more relevant to that keyword, and usually rank higher. However, it is possible to go too far and actually abuse the way a spider works.
While it is critical that you optimize your page, you can overdo optimization ( by using too many meta keywords) which can cause the spider to derive the conclusion that you are trying to fool or spam the engine.
The Process and Importance of finding Good Keywords
The first step toward a better position on the search engines is to identify exactly what your website offers. Once you have a good idea of what that is, you need to literally compress your idea to a few words.
A search engine does not use a novel or a full-page public relations newsletter to index sites. It uses words and short phrases. You determine the words or phrases that best describe your site by looking at it from the user’s perspective. How would they find your site? What search words or phrases make sense? The importance of these choices cannot be emphasized enough.
The keywords that you choose will dictate the life or death of your website.
The keywords you choose have to be keywords that you will be able to compete with them. You can’t compete with major well-established companies that are have been using those keywords. Larger, more established companies who have been on the Internet for several years are not only much more well known, but also have reciprocal links to them from many other websites. These links give credit to the larger companies and allow them to compete on such general words--engines see these sites as a reliable source. That credibility is established over a long period of time through affiliate programs, alliances, link partnerships, and other similar programs. You can not compete on those words.
The first step, then, in choosing the right keywords, is to make them specific. You can also be more specific in your keywords by narrowing down your geographic location. For example, if you have a lake Arrowhead in Georgia, you would be most interested in attracting web site visitors from the Georgia area. Georgia Internet traffic would also be more likely to visit you, rather than a Lake Arrowhead in another state. So you would be best off picking keywords like:
- Lake Arrowhead GA
- Lake Arrowhead Georgia
- Ga Lake Arrowhead
As a small player, you also have an excellent chance at ranking well on keywords that are closely related to the main subject of your site. You have a better chance of getting an excellent position using focused and targeted keywords. There are other opportunities as well, such as taking advantage of misspellings. Remember, getting a number-one listing on a misspelled word can be very easy and rewarding in terms of traffic.
Deciding on the correct keywords can be the most critical decision you can make regarding the success or failure of a website. The keywords chosen must be specific and narrow in their meaning. Picking related or misspelled words is also a viable option. Geography should be used to your advantage as well.
Results of upgrading to the Masters program… is that in a year from now, it is possible for you to have numerous many sites generating $1,000s of dollars a month in revenues while building a strong lists of segregated buyers who you can market affiliated products to on a continual basis….
