Stephan Spencer's Scatterings

The Scattered Wisdom of a scientist turned web marketing virtuoso

September 2008
S M T W T F S
 << <   > >>
  1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30        

Good cloaking: straight from the search engines' mouths

I'm here at Search Engine Strategies Chicago, and today at the "Meet the Crawlers" session I asked the distinguished panel of representatives from the four major search engines the question:

What is your current official position on simplifying the URLs selectively for bots like Googlebot, Yahoo Slurp, etc. by user-agent detection in order to drop session IDs and other superfluous parameters from the URL? Do you consider it cloaking? And if so, is it good cloaking or bad cloaking?

The panel, which included Ramaz Naam from MSN Search, Tim Mayer from Yahoo!, Charles Martin from Google, and Kaushal Kurapati from Ask Jeeves, gave me and the audience their definitive answer. But before they did, Ramez from MSN Search asked for clarification:

Will the same page content display to the user if that user types into their browser the URL that was given to the bot?

I responded with a "Yes," then all four search engines all confirmed individually:

No problem.

Then Charles Martin from Google jumped in again with:

Please do that!

So there you have it. Whether or not you call this technique cloaking or not, the search engines don't mind it, and in fact encourage it!

Posted by Stephan Spencer on 12/08/2005 | Permalink

Comments (7)| Comments RSS | Filed under: Search Engines cloaking, ip delivery            

4 comments, 3 pingbacks

  1. Actually that's old news, but since dupes became really painful it's good to bump it. Overdone white hat SEO 'ethics' got so many sites tanked ...

    Comment by Sebastian [Visitor] Email · http://www.smart-it-consulting.com/article.htm?node=148&page=103 — 12/09/05 @ 04:02


  2. No surprise there. Did anyone actually think that was something one shouldn't do?

    Comment by Jill [Visitor] Email · http://www.highrankings.com/ — 12/10/05 @ 13:53


  3. [...] СтÑ?фан СпенÑ?ер (Stephan Spencer) на конференции Search Engine Strategies Chicago, задал вопроÑ? предÑ?тваителÑ?м четырёх крупнейших поиÑ?ковиков (Google, Yahoo, MSN Search и Ask Jeeves) об официальной позиции их компаний отноÑ?ительно URL, Ñ?пецильно упрощённых именно длÑ? поиÑ?ковых ботов (здеÑ?ÑŒ речь идёт в первуў очередь об удалении идентификаторов Ñ?еÑ?Ñ?ий и других техничеÑ?ких параметров из URL). Задав наводÑ?щий вопроÑ?, увидит ли пользователь, набрав в Ñ?воём броузере Ñ?тот URL, тот же Ñ?амый контент, что и бот поиÑ?ковика, и получив утвердительный ответ, вÑ?е четыре предÑ?тавителÑ? дружно ответили “Ð?ет проблем!”. И даже больше, предложили иÑ?пользовать Ñ?тот приём чтобы не заÑ?орÑ?ть базы поиÑ?ковиков Ñ?Ñ?ылкаим на один и тот же документ Ñ? разными идентификаторами Ñ?еÑ?Ñ?ий. [...]

    Pingback by СоÐ?оты » Blog Archive » Белый клоакинг [Visitor] — 12/10/05 @ 15:29


  4. Hi Jill. This is something we've been doing and advising clients to do for a long time. I've heard SEOs say that this is cloaking and that cloaking is dangerous, so I just wanted to set the record straight.

    Comment by Stephan Spencer [Member] Email — 12/11/05 @ 16:47


  5. How about a site that uses a form to set a cookie which is then used to customize page content? how do spiders handle that or what is the recommended workaround? we use no sessions on our site so we can't store it in sesisoin. thanks.

    Comment by andrew [Visitor] Email — 03/08/06 @ 00:40


  6. [...] So, if you must use the URL’s query string to track your referral sources, then at least make sure that you aren’t ever serving those links to search engine spiders. Drop the referral source from all links when spiders come to visit. Don’t worry; the search engines say this sort of “cloaking” is totally okay. [...]

    Pingback by The Problem with Embedding Tracking Codes in your URLs @ Stephan Spencer’s Scatterings [Visitor] — 05/08/06 @ 04:31


  7. [...] in December 2005, the four major engines went on record at Search Engine Strategies Chicago to define the line between cloaking for good and for evil. From [...]

    Pingback by Good Cloaking, Evil Cloaking & Detection [Visitor] — 03/29/07 @ 03:19


Leave a comment


Your email address will not be revealed on this site.

Your URL will be displayed.
(Line breaks become <br />)
(Name, email & website)
(Allow users to contact you through a message form (your email will not be revealed.)