Redirects and SEO Best Practice
Use of the proper kind of redirects is a matter of SEO "best practice". Every site needs to have redirects (e.g. for example, from your non-www version of your site to your www version or vice versa). And if you don't, you're leaving money on the table. In addition, sites evolve over time and URLs change. And any time you make changes to your URLs -- whether it's to the domain, subdomain, subdirectories, filenames, or query strings -- you need to ensure links pointing to the old URLs are still valued by Google and the other engines, and that their voting power gets transferred to the new URLs. You can get very sophisticated with your redirects for SEO purposes and utilize what are known as "conditional redirects" -- a touchy subject and a potentially dangerous area that I'll delve more below.
First off, ensure you're using "301 redirects" rather than "302 redirects" or the link juice (PageRank) won't transfer to the destination URL. You can verify that 301s (not 302s) are in place by using a "server header checker" like this one. Only a 301 tells engines the previous URL has moved permanently and thus forwards the page's link equity to the new location.
When should you use a 301 redirect? Say you change your domain name, move content from one subdirectory into another, or you are retiring some pages. Or perhaps you are changing your content management system (CMS), which will change all of your URLs. The precious PageRank these pages have earned is valuable and you want to pass it along to the new pages. You also want to prevent a few weeks of 404 errors in search engines when no redirect is in place and users click on your listing. You'll want to do it even if you are "retiring" certain pages to an archive URL (e.g., the current year's Holiday Gift Guide once the holiday buying season is over—although I'd make the case that you should maintain such a page at a date-free URL forever and let the link juice accumulate at that URL for use in future years' editions and NOT redirect at all). If you're retiring a product, redirect the URL of the discontinued product page to that product's category page.
Duplicate content is a big issue for SEO, and 301 redirects come in very handy in eliminating (or at least ameliorating) that issue. Search engines will aggregate and filter out from the SERPs content that it detects to be duplicate. Worse yet, your pages will suffer "PageRank dilution," where the votes (links) are spread across all the duplicate versions instead of all aggregating to the one single, definitive, "canonical" URL. This can happen, for example, when tracking codes are appended to a URL (e.g., "?source=SMXad"). A current example of duplicate copies of pages with tracking code appended URLs getting indexed in Google can be found here -- ironically, it's Google's own site (yes, it happens to the best of us, even to Google!). Or when essential parameters are not always ordered in a consistent manner (e.g., "?subsection=5§ion=2" versus "?section=2&subsection=5"). Or when multiple domains or subdomains respond to the request with the same content but no redirect (e.g., "myonlinestore.com/jcp/default.aspx" and "www1.myonlinestore.com/jcp/default.asp" and "mystore.com/jcp/default.asp". In all the above cases, 301 redirects pointing to the canonical URL would collapse the duplicates and aggregate PageRank.
<START GEEK SPEAK>
Usually a single redirect "rule" can be written to match against a large number of URLs. This is referred to as "pattern matching," and it allows you to use wildcards (such as the asterisk character) and to capture a portion of the requested URL in memory and to utilize it later in the redirect. This is possible whether you are running Apache or Microsoft IIS Server as your web server. Consider some of the above-mentioned examples, and how to handle each of them using Apache's mod_rewrite module (which comes bundled with Apache):
# Changing domain names
RewriteCond %{HTTP_HOST} tiredoldbrand\.com$ [NC]
RewriteRule ^(.*)$ http://www.newbrand.com/$1 [R=301,QSA,L]
# Removing tracking parameter (but tracked URL still registers in the analytics). Assumes no other parameters.
RewriteCond %{QUERY_STRING} ^source=
RewriteRule ^(.*)$ $1 [R=301,L]
</END GEEK SPEAK>
Above I mentioned the "conditional redirect." And yes, it comes with a warning: it could get you in big trouble with Google. Matt Cutts, the head of Google's webspam team, advised during his keynote at SMX Advanced that folks not employ conditional redirects due to the risk of a Google penalty or ban. This type of redirect refers to serving a 301 redirect selectively to search engine spiders like Googlebot. Obviously, when you start serving up different content to humans than you do to spiders (and yes, this includes differing redirects), you get into dangerous territory with the search engines. If you're using conditional redirects, you probably don't even need them. For example, if you use a tracking parameter in your URLs to differentiate clickthroughs on links leading to the same content, you don't need to use conditional redirects to collapse the click-tracked duplicates and aggregate PageRank. You can treat bots and humans equally by serving them both up a 301 redirect. The clicktracked URL will still register in your log files, even though it shows up for only an instant.
Intrigued by all this and want to learn more? Then read my full article "Redirects: Good, Bad & Conditional" published last week on Search Engine Land, and also check out my PowerPoint deck from my SMX West presentation on "Unraveling URLs and Demystifying Domains".