Shrushti – SWS blog

Google: Cross-Domain Canonical Tag This Year

Duplicate content is a common occurrence on the web and in many cases can hurt search engine rankings. While the search engines may not always technically penalize webmasters for duplicate content, there are still a lot of ways it can hurt.

WebProNews is covering the Search Marketing Expo (SMX) East in New York, where representatives from the three major search engines (Google, Yahoo, and Bing) discussed how their respective web properties handle duplicate content issues. Following are some takeaways from each.

Duplicate Content in Google

Duplicate Content on Google – Joachim KupkeThe way Google handles duplicate content has been discussed a lot in recent memory. This is largely due to a video Google’s Greg Grothaus uploaded, in which he discusses at length, the way Google handles a variety of different elements of the duplicate content conversation.

Joachim Kupke, Sr. Software Engineer of Google’s Indexing Team reiterated much of what Grothaus said. He also said that Google has a ton of infrastructure for content duplication elimination:

- redirects
- detection of recurrent URL patterns (the ability to ‘learn’ recurrent url patterns to find duplicated content)
- actual contents
- most recently crawled version
- earlier content
- contents minus things that don’t change on a site

Kupke said to avoid dynamic URLs when possible (although Google is “rather good” at eliminating dupes). If all else fails, use the canonical link element. Kupke calls this a “Swiss Army Knife” for duplicate content issues.

Google says the canonical link element has been tremendously successful. It didn’t even exist a year ago, and is has grown exponentially. It has had a huge impact on Google’s canonicalization decisions, and 2 out of 3 times, the canonical tag actually alters the organic decision in Google.

Google says a common mistake is designating a 404 as canonical, and this is typically caused by unnecessary relative links. So, avoid changing rel=”canonical” designations, and avoid designating permanent redirects as canonical.

Also, do not disallow directives in robots.txt to annotate duplicate content. It makes it harder to detect dupes, and disallowed 404s are a nuisance. There is an exception however, and that is that interstitial login pages may be a good candidate to “robot out,” according to Kupke.

Kupke says that canonical works, but indexing takes time. “Be patient and we WILL use your designated canonicals.” Cleaning up an existing part of the index takes even longer, and this may leave dupes serving for a while despite rel=canonical, Kupke adds.

At SMX, Google announced that cross domain rel=canonical is coming within this year. So for example, if the Chicago Tribune has an article on the New York Times, and the rel=canonical points to the Chicago Tribune then Google will only credit the Chicago Tribune with the content.

Duplicate Content in Bing

Sasi Parthasarathy

As far as how Bing views duplicate content, intention is key. If your intent is to manipulate the search engine, you will be penalized.

Sasi Parthasarathy, Program Manager of Bing says to consolidate all versions of a page under one URL. “Less is more, in terms of duplicate content.” If possible, use only one URL per piece of content.

Bing isn’t supporting the canonical link element (as a ranking factor) yet, but it is coming. They do say to use it, but it’s just not really a ranking factor in Bing yet. Bing says that there has been an increase in the usage of canonical tags in the past 6 months, but adoption issues still exist. According to Parthasarathy, 30% of canonical tags point to the same domain (which is fine), and 9% use it to point to other domains. This could be a mistake or it could be manipulative. Bing says they will look for other factors to try and determine which it is.

Bing says canonical tags are hints and not directives. “Use it with caution,” and not as an alternative to good web design.

With regards to www vs non-www, just pick one and stick with it consistently. Remove default filenames at the end of your URLs. Bing also says 301 redirects are your best friend for redirecting, use rel=”nofollow” on useless pages, and use robots.txt to keep content you don’t want crawled out.

Duplicate Content in Yahoo

Cris Pierry

If everything goes according to plan, you’re going to need to worry about how Bing handles duplicate content if you’re worried about how Yahoo handles it, but Yahoo’s Cris Pierry, Sr. Director of Search, offered a few additional tips.

Pierry says descriptive URLs should be easily readable, and it’s not a good idea to change URLs every year. In addition, use canonical, avoid case sensitivity, and avoid session IDs and parameters.

Pierry also says to use sitemaps, and submit them to Yahoo Site Explorer. Improve indexing by proper robots.txt usage, and use Site Explorer to delete URLs that you dont’ want Yahoo to index. Finally, provide feeds to Yahoo Site Explorer, and report spam sites linking to you in Site Explorer.

Yahoo says metadata and SearchMonkey are enhancing presentation.

Source: http://www.webpronews.com/topnews/2009/10/05/duplicate-content-on-google-bing-yahoo

Hi,

During working on two different sites in different area with same age,  I fill that the original content wrote by us in our website provide a good value, but how to know after some period or some amount of time they are  powering our results or decreasing our results. Do you know how search engines are calculating the are of document ? ( I am not talking about the website age) Due to ever changing search industry and competitiveness of the type of business we need to know how this things are calculated..

Let us discuss the few common factors considered by the search engines to count a age of document when you upload or create a new data / document or fresh page on your website, How they know that this page is a new page ..

• When they first crawled a data on the domain where you upload the new created page.
• They found referene of your newly created page on some document or on a good article or in some emails or on a page with good content.
• By combination of both the ways simultaneously.
• The date and time on hosting server.
• The Document may be determined from the date that search engine first discovers a link to the document.

Its good most of the time that how fast search engines know that the new content on your websites just born. Because of the nature of some nature of industry, NEW content or news is more important then old documents.

If you want some document to be appear on search engine for some period of time, then you can do by providing NO Follow attribute in meta tags of the document. Still you can optimize them with link building and directory submission without crawling your data with no follow.

I think I am taking too much technical SEO. We will continue with the next article.