Monitoring Social Media for Breaking News, Events

Hello from the Web 2.0 Expo in NYC. Yesterday I presented “Best Kept Secrets to SEO Success”, where I shared some of lesser known tricks, tools, and techniques.

One of the sessions I attended after mine was finished was “Monitoring Social Media for Fresh, Contemporary and Reliable Sources” presented by Andrew Baron, founder of the popular Rocketboom videoblog/podcast. Andrew shared some of the ways he keeps his ear to the ground on social media to pick up on breaking news before it hits the mainstream media. He uses Search.Twitter.com (formerly Summize) to search for what’s going on at this instant. For example, he was able to find that the line at the Apple Store was around the block, saving himself a trip to the Apple Store (he went later after the line died down). A lot of news breaks on Twitter, so tracking Twitter not just through searching but also by following people and watching their tweets is critical. Andrew pointed out the list of the 100 most followed Twitter users on Twitterholic as notable folks to track. (I also recently discovered Twubble which helps you find new Twitter friends based on your current network of Twitter friends – pretty cool.) Andrew really likes Friendfeed; he referred to it as “Twitter with a real conversation.” The most popular links shared via Friendfeed are on Friendfeed Links. Some of the “most popular” links and “fast movers” across numerous social media are easily tracked in one place using popurls. Wikipedians are frequently first on the scene to provide late-breaking news – scurrying to update the relevant Wikipedia articles as things unfold; these can be tracked with Wikirage. Narrow the results down to within the hour for the really fresh stuff. When there are a lot of recent updates to an article it can indicate something interesting is happening. Andrew provided cautionary tales of the Montauk monster hoax and the Amazon rainforest tribe that was untouched by the outside world that was real, then a hoax, then finally real again to make the point that you have to analyze very critically the validity of any story found on social media. Andrew quoted Paul Virilio, a French philosopher, to make the point that whoever acts on new information first has an advantage. This holds true in many situations – on the battlefield, on Wall Street, and in the Web 2.0 world of startups. So if you have something new, take action before it becomes old.

Natural Search Tactics for Retailers

On Tuesday I gave two presentations at the Shop.org Annual Summit. First was “Natural Search Tactics for the Retailer” with fellow panelists Ken Jurina from Epiar, Jenny Schlueter from Dell, and Ian McAnerin from McAnerin International. It was a very tactical session – focused on tools, tips and techniques. Ken covered keyword research, Jenny covered content optimization, Ian covered technical optimization, and I covered link building.

Some are a few key points from the session…

• Mine multiple data sources for keyword data, such as Google Insights for Search, WordTracker, Yahoo Panama Keyword Tool, Google Keyword Tool, MSN Keyword Forecast, Trellian Keyword Discovery, Google Webmaster Tools, Wordze, Google Suggest, Google Traffic Estimator, Google Trends, Hitwise, NicheBOT, SEO Book Keyword Tool, GoodKeywords.com, internal site search logs, referrer logs, and PPC broad match. I’d add to Ken’s list: comScore Marketer, WordPot, and Yahoo Search Assist (the autocomplete that is built into the Yahoo search box).
• Determine keyword difficulty with the SEOmoz Keyword Difficulty Tool, InternetMarketingNinjas Top Ten Analysis Tool, and KEI from KeywordDiscovery or WordTracker.
• Useful link analysis/link building tools include Yahoo Site Explorer, Back Link Analyzer, Thumbshots Ranking tool, TouchGraph, SEOChat PageRank Lookup, SEOChat PageRank Search and SEO for Firefox extension.
• Types of links that are likely to get discounted include: reciprocal links, affiliated sites (on the same IP range or hostname), footer links (at the bottom of the page), site-wide links, links contained on a page called links.htm / links.asp, and links with the exact same anchor text. Remember that the more links on the linking page, the less PageRank you’ll get.
• Review your existing links using the Back Link Analyzer and contact those webmasters who link to you with suboptimal anchor text. Focus on the highest value links where you have rapport or influence with the webmaster.
• If you have multiple servers, rotating IP addresses (load balancing) can make it look to Google like you have duplicate copies of your site, and edge computing can cause geolocation issues. The fix is to detect spiders and send them to a canonical site version.
• Use the country code TLD for a country, use a subdomain for language or major group, and use subdirectories for topics. e.g. language.company.ccTLD/topic/page.htm
• A gTLD (.com, .net) is almost always geolocated via IP address. A ccTLD (.ws, .la, .tv) overrides IP geolocation. The duplication problem does not affect clearly geolocated sites. Always declare language and character types on web pages.
• Scripts that build links on the fly, AJAX, and Flash are bad for SEO. An Iframe is treated as a separate page; includes are better. Use a spider simulator such as SEO-Browser to test your site.
• Search engines strip most HTML code out of a document before parsing, so most HTML validation errors do not affect rankings. Exception: some errors, such as a missing “<” can kill the indexing of your page. Best Practice is to always validate your code.

The Powerpoint, which includes all 4 presentations, is available for download here.

The second session I presented was with Amy Africa and it was a site clinic session where Amy and I did impromptu critiques of audience members’ websites. Amy covered usability and conversion; I covered SEO. It was a lot of fun. There were no Powerpoints for that session.

DishyMix Podcast Interview

I was interviewed by Susan Bratton for the DishyMix podcast several weeks ago while I was in San Jose for SES. That interview is now live. It was a lot of fun. Listen to it here. There’s also a transcript available. We talked about SEO (particularly for blogs and bloggers) and a range of social media. Enjoy!

Leveraging Your Affiliates for PageRank

Affiliate networks like CJ use 302 redirects, even though they know that blocks the flow of PageRank to the merchant. And that’s not going to change anytime soon — if ever — because the networks’ loyalty is to the affiliates, not to the merchants. If merchants profited from affiliate links in terms of rankings improvements, the affiliates would not be happy. There would be mass revolt and a bunch of affiliates would leave the network. Luckily, for you merchants, there are a couple workarounds you could utilize to capture some link juice from your affiliates…

1. Bring your affiliate program in-house and utilize an affiliate tracking solution that uses 301 redirects and therefore passes PageRank. Here’s a blog post of mine about this: Affiliate Programs That Pass Link Gain (PageRank). Amazon.com is one such merchant with an in-house affiliate program — and it serves up 301s to capture PageRank from their “associates” (affiliates).
2. and/or, simply require your affiliates to post a disclosure statement on their Legal Notices page (or their About page if they don’t have a Legal Notices page) stating that they are an affiliate of the merchant and that neither party is an agent, partner, joint venturer, franchisor, franchisee, employer or employee of the other. And of course require that the affiliate include a straight link to the merchant in that statement. Pretty sly, eh! And the great thing about a Legal Notices page is that it is typically linked site-wide and has very few other links on it compared to other pages, so your link gets a bigger slice of the PageRank that is divvied up among the links!

Bear in mind that affiliates can always add rel=nofollow to any and all links to the merchant, so the power still rests with the affiliates.

My Favorite Command Line “Hacks” – for Power Users (Geeky!)

I’m on a Mac and I love working from the command line (i.e. from within the Terminal or iTerm). Here is a random collection of some of my favorite “hacks” (time-saving tips and tricks), which will also work not just on Mac OS X, but in Linux/Unix distributions such as Ubuntu and with whatever shell you prefer (bash, csh etc.)…

(Please note: I’ve assumed you have Perl installed and in your PATH. If you don’t know what I’m talking about, these tricks are not for you, and you’ll probably do some damage. You have been warned. )

Search and replace within a file – without launching a word processor!

Hypothetical example, replaces all occurrences of “chicken” with “fish” in a text file called somefile.txt and saving a back-up (just in case!) to somefile.txt.bak:

perl -pi.bak -e 's|chicken|fish|g' somefile.txt

Real-world example, using regular expressions. Replaces all dates in a dd/mm/yy (British date) format in a Quicken QIF file (that I downloaded via online banking from a bank in New Zealand) with American date format mm/dd/yyyy for use in the American version of Quicken:

perl -pi.bak -e 's|(\d\d)/(\d\d)/07$|\2/\1/2007|g' download.qif

Another advanced example, that takes all the .html files in the current directory and strips out anything that was commented out using comment tags, i.e. <!– –>…

perl -pi.bak - 0777 - e 's|<\!--.*?-->||gs' *.html

Rename a bunch of files in a directory – in batch!

This example renames all files in the current directory ending with .jpeg so they end with .jpg instead:

for i in *.jpeg; do; mv $i `echo $i|sed 's/jpeg/jpg/'`; done

Backticks (`) are amazingly powerful. With them you can execute commands and have the output of that command get used in another command. Yep, nested! Awesome!

Batch resize images

This example creates thumbnail images of all the .jpeg files in the current directory, each being no bigger than 400 pixels in either dimension, and the height-width aspect ratio is kept intact (i.e. the image is not stretched). Each thumbnail filename is prepending with the word “small”, followed by the original file’s filename (e.g. from lolcat.jpeg to smalllolcat.jpeg):

(Note that this hack requires you to have the ImageMagick package installed, which includes the very handy “convert” command. To check whether ImageMagick is installed, type “which convert”, then if nothing turns up, try “locate mogrify” (I didn’t suggest “locate convert” because that would probably turn up a lot of other apps and files besides ImageMagick’s convert). The easiest way to install ImageMagick on the Mac is to download a precompiled binary pkg file.)

for i in *.jpeg; do; convert -geometry 400x400 $i small$i; done

(Tip: This is especially handy for when you want to upload a bunch of images to Flickr but don’t want to waste bandwidth and time uploading the high-resolution originals.)

Batch convert the image format on a bunch of files

This one takes all the .jpg files in the current directory and creates .png files out of them. You could alternatively convert pngs to jpegs, gifs to jpegs, jpegs to gifs, gifs to pngs, etc. etc. Each filename has the same base as its counterpart (e.g. from lolcat.jpg to lolcat.png).

(Note that this also requires ImageMagick.)

for i in *.jpg; do; convert $i `echo $i|sed 's/jpg/png/'`; done

Examine the type of redirect (301 or 302) on a URL and where it leads – great for following a chain of redirects!

Sure, you could do this with the livehttpheaders extension but I don’t use Firefox, I use Safari ‘cuz it’s so blazingly fast on the Mac compared to Firefox or even Camino.

(If you don’t know why you should care about which type of redirect you’re using or whether it’s a long chain of redirects, you’d better read my Search Engine Land article: Redirects: Good, Bad & Conditional.)

Here I’m looking at the hypothetical example URL of www.example.com…

(Note that you don’t need to use a properly formed URL beginning with http://.)

lwp-request -Sd www.example.com

Don’t forget to put the URL inside of quotes if the URL is a dynamic one with question marks and ampersands (or other special characters). Like so:

lwp-request -Sd "http://example.com/search?q=cheese&num=20"

More likely than not, you don’t have lwp-request installed already. If that’s the case, it’s easy to install LWP, it’s just a CPAN library. Install it like so:

sudo cpan install LWP
Enter “no” when prompted whether you are ready for manual configuration.
Then hit the Return key a bunch of times

If it can’t find your “make” program, you can configure it like so, using whatever your path is to make (in the example below I’ve assumed /usr/bin):

perl -MCPAN -e shell
o conf make /usr/bin/make
install LWP

Back up your MySQL database to a SQL file – especially before running a WordPress upgrade!

I’ve used the hypothetical database name of “databasename” in the example below:

mysqldump -udba -p databasename > databasename.sql

Consider adding this command to your crontab if you want to have regularly scheduled database backups (crontab -e for that).

Back up your folders using rsync

Sure, if you’re a programmer you probably live your life within a version control system like Subversion (svn). And so for you it will be trivial to put your important documents in a Subversion repository and under version control. But I don’t live and breath svn. I know enough to be dangerous – and I intend to keep it that way! (I practice Tim Ferriss’ philosophy of “selective ignorance“.)

If you’re lucky enough to own a Mac, just buy a Time Capsule and be done with it. But for the rest of you, you can use rsync to remotely sync (back up) your computer.

Here’s an example of backing up your Documents directory onto a remote server:

(Note: This assumes you’ve set your domain name using “domainname” or you have “Search Domains” in your Network settings set to your local domain name. Otherwise use the full hostname with the domain name in the following command, e.g. eeyore.netconcepts.com instead of just eeyore.)

rsync --force --ignore-errors --delete-excluded --delete -av -z -e ssh ~/Documents root@eeyore:/home/stephan

Here’s an example of how you can back up your mail folder from the mail server onto a local backup drive:

rsync --force --ignore-errors --delete-excluded --delete -av -z -e ssh root@eeyore:/home/stephan/mail /Volumes/BackupDrive

View the code in a binary file

Strings shows only the text that can be extracted from the binary file. I use it when I don’t want to launch Microsoft Word (a memory hog that takes 30 seconds to start up on my machine) but I want to quickly view a Word document’s contents, like so:

strings something.doc

If I need to examine the data within a binary file, byte by byte, I use od (stands for octal dump) rather than strings. Like so:

od -c something.exe

od is also handy for examining the ASCII codes of strange characters in a HTML page, like so:

lwp-request http://shop.bellacor.com/results-13/70/1.htm | od -c

History repeats itself

I use the up arrow key all the time to pull up previous commands so I can revise them and reissue them. Once the chosen command is displayed, I can use backspace, delete, etc. to change it. An even better way to do it is to use the ^ operator if you simply want to swap out one word for another, like if you made a typo.

For example, if the previous command entered was a typo:

covert -geometry 200×200 big.png small.png

Then you could correct that simply by typing:

^covert^convert

Cool eh!

I love referring back to my history and re-using previous commands. Sometimes I want to scan all my history, for that just type:

history

Another huge timesaver is the exclamation point. But do yourself a favor and refer to it as the “bang” operator. So if you were to spell out out !con to your geek friend over Skype, you would say: “bang c-o-n”. That’ll make you sound like you’re in the inner sanctum of all geeks.

Here’s how it works… Bang followed by a number executes the command at that specified line number in your history output. For example:

!127

Bang followed by letters executes the command in your history that begin with the specified letters. So if I wanted to rerun the last ping command I ran a while ago, I’d just type:

!ping

Or if I’m extra lazy I might type:

!pin

I wouldn’t type !p because I could accidentally issue an unintended command if I had forgotten I had used more recently and that also happened to start with a p.

What about QuickSilver, you ask?

If you’re a Mac power user, you’re probably using QuickSilver and getting big productivity gains from it. Well good for you! I myself don’t use it that much, primarily just for launching applications. But I plan to use it a lot more. I just have to get myself into that habit. If you’re a Mac user and want to learn more about QuickSilver, the best introduction to it (imho) is this video of Merlin Man demonstrating it on MacBreak. I don’t see the bash shell (Terminal) and QuickSilver as mutually exclusive. Really they go hand-in-hand.

Affiliate programs that pass link gain (PageRank)

Most affiliate programs do not benefit search engine rankings because the link from the affiliate to the merchant doesn’t count as a “vote.” Thus, the merchant will not see a benefit in their Google PageRank and consequently in their search engine rankings. For example, any merchant using LinkShare or Commission Junction will not see such a benefit. That’s because they all use temporary redirects, also known as 302 redirects. That type of redirect, which is the one programmers and site administrators tend to use by default, doesn’t pass the link gain (e.g. Google PageRank) on to the target (final destination) URL. Only a very few affiliate management services allow the merchant to capitalize on the link gain of the affiliate. MyAffiliateProgram.com is one such affiliate solution. So I checked them out, and it turns out that it kinda works. Yes, kinda.

Here’s the problem. The affiliate solution needs to use permanent redirects (a.k.a. 301 redirects) rather than temporary (302) ones. MyAffiliateProgram.com uses what they call “direct links.” Here are a couple examples of affiliate-tracked direct links that they provided me to look at: http://www.myaffiliateprogram.com/?kbid=1001 or http://www.kitchen-universe.com?kbid=1001. But when you visit either of these 2 URLs, there is no redirect at all. Consequently, this creates lots of duplicate pages in Google when Googlebot finds these affiliate-tracked direct links and follows them. Taking the first URL as an example, if you search Google for site::www.myaffiliateprogram.com inurl:kbid you’ll see 6,980 duplicate pages in Google. In other words, these are pages that were already in Google with URLs that don’t have kbid= appended at the end.

Think about it this way: Yes, with MyAffiliateProgram.com a merchant will get PageRank flowing to all the links contained on the countless duplicates of the merchant’s home page that are getting indexed. But because there is no 301 redirect present, MyAffiliateProgram has failed to collapse the link gain to one definitive version of the merchant’s home page. Then search engine spiders come along and index all these versions of the merchant’s home page which compete with the merchant’s true home page (the one without any kbid=). Furthermore, searchers who click on listings in the search results that contain kbid= in the URL will get counted as referrals from the affiliate and the merchant will pay for that. Ouch!

So, buyer beware when shopping for an affiliate management service that passes PageRank to your site. The devil’s in the details.

Any readers want to recommend affiliate solutions that do effectively pass link gain?

UPDATE: Just found this great blog post from Greg Boser that discusses this issue in more detail.

Our Web 2.0 corporate website

It is amazing what you can achieve in regards to SEO just by reaching into your Web 2.0 toolkit. A corporate brochure is about as Web 1.0 as you can get, yet even brochureware can be transformed in ways you may never have thought possible and boosting search visibility in the process.

To prove that point, we redesigned our Netconcepts.com site and launched it this month. It’s a typical corporate website, with a portfolio, testimonials, case studies, an article library, bios of our executives, information on our services, etc. The redesigned site is now powered by WordPress, a popular blogging platform.

Because it is now qualifies as a blog, our corporate site can start to enjoy some visibility in the blog search engines and directories and, because WordPress comes with RSS feed capability built in, we can now start to enjoy some visibility in the feed search engines and directories too. We don’t just have one RSS feed, by the way. We have many, grouped by topic (e.g. SEO, email marketing) and by type of resource (e.g. articles, testimonials), done through the use of tagging.

Speaking of tags, that is where the real SEO magic happens in this corporate site because every testimonial, every portfolio entry, every press mention, as well as each bio, article and case study, is broken out into a separate post. That post is tagged with appropriate keywords, for example all the testimonials are tagged with the word “Testimonials”. So instead of having a single testimonials page as we used to, we have a testimonials tag page that spans three web pages (at 10 posts per page) and each of the 30 testimonials is a separate web page now too. In other words, we want from 1 page to 33 pages; that’s a lot more search engine fodder, all with different keyword foci!

Spiders can find and index these tag pages through the text links contained within the tag cloud on the home page, and through text links underneath each post, and through links to “Related Tags” on each tag page. Related Tags are determined from posts that have the tag (from the tag page in question) in common. So, for example, because we have posts that are tagged with both “SEO” and “Testimonials”, therefore “SEO” appears as a related tag on the Testimonials tag page and “Testimonials” appears as a related tag on the SEO tag page. We display links not just to the related tag pages, but to intersections between related tags. So, for example, you will find on our Testimonials tag page a number of Related Tags in the right hand column, all with “AND” and “OR” links adjacent to each one.

Let me restate that a little bit differently just to clairfy… All our SEO-related items (testimonials, case studies etc.) are tagged with “SEO”. Consequently, there is a tag page that relates to “SEO” and a tag page that relates to “testimonials”. There’s even a tag page that relates to “SEO testimonials” — the intersection of those two tags. That makes for boatloads of tag pages, considering how many different permutations there are for various combinations of tags being “ANDed” or “ORed” together.

By moving to a more modular structure (based around Posts rather than Pages) along with the large inventory of new pages, allows us to capitalize on the “Long Tail” of natural search in ways we couldn’t dream of with our previous incarnation of brochureware.

Above the “Related Tags” you will see a RSS button which leads to a RSS feed specific to that tag page. We use that RSS feed to pull the latest articles, testimonials, seminars etc. and feature them as related content on the right column of our Services pages. For example, our Email Marketing page lists Related Articles on email marketing, because we’ve specified that the topic of the page is “email marketing” (in other words, the tag that it relates to). We use that information to grab RSS feeds of tag pages for email marketing + articles, email marketing + testimonials, etc.

Besides tagging, we have also employed many other blog SEO tactics, a number of which I detailed in my article for MarketingProfs, 10 Tips to Help Your Blog Soar in the Search Engines. This includes use of sticky posts, adding buttons to add a post to del.icio.us and various other social bookmarking services, and linking to a “Top 10″ list of sorts — namely, under “Free Stuff” on the home page, some of our best content from the past year or so.

To my knowledge this approach for search engine optimizing a corporate site has not been done before, particularly the aspect of breaking up all the discrete bits of content (each testimonial, each portfolio item, each FAQ, etc.) into individual posts and tagging all them, and then relating that tagged content with the appropriate Services pages and highlighting those as related content. If you have heard of a corporate site doing this, please let me know. I would love to check it out.

I also welcome any feedback on what we have done here on the Netconcepts site. We still have a few issues to tidy up with the site (so don’t expect perfection), yet overall I am quite pleased with how it all came together.

And, last but not least, the new redesigned site, although it doesn’t look markedly different from our old site, is better designed with XHTML and web standards in mind. No more tables for layout. Yay! That was long overdue. Now maybe we will see a rankings benefit in Google Accessible Search.

Squandering Your Crawl Equity: Don’t Do It!

A couple weeks ago I spoke at Search Engine Strategies San Jose on Long Tail SEO Tactics (here’s my Powerpoint, btw, if you’re interested) and at SEOMoz Expert Training on the topic of Site Architecture and Internal Linking (download Powerpoint), and I found that I was addressing the same key issue to both audiences. That issue is…

How do I best spread my PageRank (link juice) across my (very large) site?

This question could be restated as:
How do I avoid squandering my crawl equity?

What do I mean by “squandering crawl equity”? Well, how deeply Googlebot goes into your site will depend in part on the PageRank, trust and authority you have earned (in the eyes of Google). It will also depend on how spider-friendly your URL structure and internal linking structure is. You can squander some of that crawl equity by presenting a plethora of low-value pages or duplicate content to the spider.

Often times this is done inadvertently. Consider the following “spider trap” scenarios, all of which are undesirable from Googlebot’s standpoint:

1. Every page of content has an “Email this page to a friend” link that leads to a page with a form (and optionally some duplicate content that was also present on the canonical page). The link to the form page is followable (i.e. no use of rel=nofollow) and the form page itself has a unique URL that is accessible to Googlebot (i.e. not disallowed by robots.txt directives or noindexed by meta robots tag). This is the case on Biolcell.org, but it occurs on millions of other sites too.
2. Pages of internal search results (i.e. generated by the site’s search engine) are accessible via links, and then from there, links lead into the pagination structure (i.e. Next and Previous as well as page number links such as page 9 of 51), as well as to results pages for searches on numerous “related” keywords. This creates rampant duplication of content as can be seen in this example.
3. Lists of “Related Tags” are shown on the site and these lead to tag conjunction pages where some or all of the tags are OR’ed together. For example, here. This example is taken from my own company’s website Netconcepts.com — and I’m pleased to say that our tag drill down feature has the “OR” links nofollowed to avoid sending spiders down an infinite loop of duplicate content.
4. Faceted navigation (attribute-based navigation) creates seemingly limitless permutations by offering filtering and sorting options through crawlable links. Torrey Hoffman of Google’s Webmaster Central team presents the following ecommerce example in his recent Google Webmaster Central blog post:
   

   Another common scenario is websites which provide for filtering a set of search results in many ways. A shopping site might allow for finding clothing items by filtering on category, price, color, brand, style, etc. The number of possible combinations of filters can grow exponentially. This can produce thousands of URLs, all finding some subset of the items sold.

   Faceted navigation, when implemented without expert SEO guidance, can sabotage your site’s SEO. We’ve seen search-delivered traffic tank on more than one occasion when a retailer implements faceted navigation “out of the box” without retooling it for SEO. You retool it with rel=nofollow on links pointing to low SEO value facets (like price range), with meta robots=noindex on the low SEO value permutations, and/or with alternative taxonomic navigation structures (with good anchor text).

   My colleague Brian delves into this thorny issue of search engine optimizing “guided navigation” in his article today on Search Engine Land. Check it out.

Catch this video of the Teen Entrepreneurs Panel at Ypulse

I shot the following video of teen entrepreneurs Chloe Spencer of the Ultimate Neopets Cheats Site (my daughter!), Juliette Brindak of Miss O & Friends, Jared Kim of WeGame and Angela McBride, a.k.a. Asuka Martin in Teen Second Life presenting on the “Totally Wired Superstars” panel (moderated by Ypulse founder Anastasia Goodstein) at the Ypulse Mashup youth marketing conference in San Francisco this past August. Fascinating how these kids can build their businesses and balance that with school, extracurricular activities, friends, college entrance applications, etc.!

Or download the video to play it on your iPod.