Relying on search engines to automatically crawl your website is not the best way to get your pages indexed. You must explicitly tell Google what pages you want it to list to potential visitors! This post will go into some detail and offer some different techniques on how to index your site more completely on Google. These techniques will be applicable to other search engines as well.

When you deal with your business finances, do you “hope” you have money in your account to pay for regular business expenses?  The chances are you understand exactly what your business costs to run, what you have in inventory, and you update your records when those bills or inventory changes.

Well, why would your website be any different?  It would seem, especially for websites that drive customers to your business (ALL websites should do this for their parent companies!), that you would have a solid inventory of the pages your site includes and the information contained on them.

The inventory that Google cares most about is called a Sitemap.  Sitemaps can have many forms, XML, URL Text lists, and RSS feeds.  Google accepts them all.  XML Sitemaps include information that tells google multiple pieces of information.

1. Page URL
2. Priority
3. Frequency
4. Last Change

Google uses each of the above pieces of information to help it generate relevant content for its users.  Google reads the first information, Page URL, to know the page that your site includes.  It now adds this page to its index list.  This allows Google to figure out what pages your site includes, so its spiders can go out and index the page for future visitors.

The second part, Priority, is a number from 0 to 1 (decimal) that tells google how important the page is in relation to the rest of your website.  This, at first, may seem strange, and hard to grasp, but is a great way of ensuring that Google spends the most time and effort indexing your most important pages.  Further, if Google knows which page you think is more important to your overall user’s experience, it can decipher between two pages that it thinks are equally relevant to a user’s search.  For example, if you run a site selling dog treats, and you have two pages that look to Google to be equally relevant to a user’s search, Google can use the information in your sitemap to show the page you identified as being more important.

Google uses the frequency option a little more lightly, since it is a term that can be set by the website owner.  This XML key allows a website owner to declare how often a particular page will change.  It will help google know how often it has to re-index the page.

Last Change tells Google when the last time the page has changed.  This is a simple key that keeps Google’s index up to date.  If it downloads a sitemap that says a page has changed since the last time Google indexed that particular page, then it knows it must go to re-index that page.

As you can see, Sitemaps are necessary for business websites to get properly indexed.  The problems it may fix include, but are not limited to:

1. Pages that are un-reachable through your current link structure
2. Pages that change frequently, and announce that they have changed through a date
3. Inventory control for your website pages

Understanding what you have exactly is a common theme in the world of business.  Treat your website just as you would treat your business inventory and finances, because your website directly affects both of those things, if you let it.

This entry was posted on Sunday, July 13th, 2008 at 9:14 pm and is filed under Accessibility, Generate Revenue, Search Engine Optimization. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.