This article was originally published under Search Engine Land.
Optimizing a website that has tens of thousandsโor even hundreds of thousandsโof dynamically generated pages, requires thinking differently. Old school SEO, where you assign each page a keyword theme based on keyword research and hand-craft a title tag, H1 tag and intro copy, then figure out the best internal links to send to the page, just doesnโt scale with big sites. Particularly when youโre talking about the magnitude that our Netconcepts clients are operating atโtypically over 10,000 SKUs and over 100,000 indexed pages.
Itโs essential that you focus your SEO efforts in such a way that the effects will cascade through your site. For example, come up with โrecipesโ for optimized titles for product pages, for category pages, for articles, etc.โyet allowing for those recipes to be overridden with a hand-crafted title tag when required. Getting the title tag right will make a big difference. For example, the website SlideShare.net has over 40,000 tag pages indexed in Google, but the titles are suboptimal. They all follow the recipe of โSlideShare ยป Slideshows tagged with [keyword].โ A better choice would have been โ[keyword] tagged PowerPoint slides, presentations and slideshows.โ Such a change is usually easy to implement and is likely to pay big dividends in rankings and traffic improvements.
Donโt stop at the title tag; optimize the entire HTML template. Use SEO best practices: 1) separate out the content layer from the presentation layer; 2) make sure youโre using semantic markup; 3) employ heading tags (e.g. H1, H2) when appropriate; 4) cut the bloat out of the template; 5) make sure youโre not using the same meta description and meta keywords across the whole template. Make that template really hum.
Then move on to your URLs. Granted URLs are harder to optimize, but itโs usually worth the effort. Particularly if your URLs have more than a couple parameters (i.e. more than two equals signs). Google engineer Matt Cutts told the audience at WordCamp this past weekend that dynamic URLs and static URLs are treated the same by Googleโwith the caveat that as long as there arenโt more than 2 or 3 parameters in the URL. Nonetheless, Iโd rewrite your URLs to remove the query string (i.e. question mark) altogether, using a server plugin like mod_rewrite or ISAPI_rewrite. If rewriting your URLs and otherwise deploying your optimizations are difficult/slow/expensive due to IT department bottlenecks or ecommerce platform/CMS limitations, there are proxy server based workarounds like GravityStream (which fellow Search Engine Land columnist Chris Smith recently described as โautomatic SEOโ). However, whenever feasible you want to fix your native site.
Itโs been our experience that static URLs perform better in the engines. As a bonus, such URLs look nicer to users so they tend to garner more links too. Ideally you should go for keyword URLs. A URL like http://www.mysite.com/kitchen-sinks.php is superior to a URL like http://www.mysite.com/product-34962.php. Matt Cutts also announced at WordCamp that underscore characters are now going to be treated as word separators. So no need to worry about whether itโs an underscore or a hyphen youโre using to separate wordsโat least as far as Google is concerned. Oh, and make sure that your old URLs respond with a 301 permanent redirect to the pageโs new, optimized URL.
I like to think of my collection of web pages indexed by the search engines as my virtual sales force. Each unique, indexed page at a unique URL is like a virtual โsalesperson.โ The more virtual salespeople working for you, the better. Unfortunately most of these salespeople are freeloaders, sitting around doing nothing for youโnot attracting a single search engine visitor. Increase your indexed pages while at the same time decreasing your freeloaders. Employing spider-friendly URLs decreases the percentage of freeloaders.
Effective tactics for adding more pages to your virtual sales force include deploying faceted navigation (such as Endecaโs โGuided Navigationโ), pulling in content through APIs (Application Programming Interfaces, such as that provided by Flickr), and leveraging your visitors as content co-creators. Your visitors can be invaluable unpaid employees for youโpopulating your site with product reviews, discussion forums posts, blog posts, blog comments, wiki articles. The great thing about user-generated content is that it incorporates your consumersโ vocabulary into your site. So even if youโre wedded to an industry buzzword (e.g. โkitchen electricsโ), you can rely on your visitors using the more popular synonym. When your visitors wonโt do your dirty work for you, turn to the โMechanical Turk, โ Amazonโs scalable human-powered service that surprisingly few SEOs utilize. Imagine an army of humans paid in micropayments to do your bidding. Mechanical Turk can tag your products, tag your images, translate your English language content, transcribe your audio, and much more. Whatever you canโt scale algorithmically, you can probably scale through the Mechanical Turk.
Encourage people to syndicate your content (and links) by providing numerous RSS feeds powered by your data, sliced and diced in different ways (most popular, top rated, clearance, newest and latest, by category, etc.). This propagates deep links into your site from blogs, aggregators and aficionado websites (and yes, from splogs tooโฆsigh!). Also prominently display and encourage visitors to use social bookmarking services such as del.icio.us throughout your site, in order to add your content to their bookmarks and tag themโagain, for the deep inlinks.
Another thing that decreases your percentage of โfreeloadersโ is your internal linking structure. Your navigational hierarchy plays a key role in passing link gain deep into your site. Pages too far down the site tree wonโt get enough โjuiceโ to warrant high rankings. Optimize your linking structure by creating a rich web of interlinking within your site. Whenever appropriate, include links to related products, related articles, related searches, etc. While youโre at it, ensure the anchor text is optimal (i.e. wipe such phrases as โview relatedโ and โclick hereโ from your link text vocabulary). Tag clouds are one of my favorite methods of interlinking with keyword-rich text links, done in an attractive Web 2.0 way. Donโt just use the same tag cloud across your site; tailor the tag cloud to the page or category within the site.
Iโve seen search engine optimization scale across very large websites through automation and delegation, rather than old school SEO tactics. Just like with most things, the secret lies in working smarter, not harder.
