#1
  1. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2004
    Posts
    72
    Rep Power
    15

    What is the best way to identify and remove duplicate content?


    I know, this topic was touched on in my other post, but I wasn't able to get a clear explaination. It appears that I am not getting alot of visitors becuase my site isn't ranking well on google because google has omitted most of the pages from is search results because google has decided we should be penalized because we have duplicate content. Follow that? Great ;)

    Anyways, I am desperately trying to sort all this out and get the site back to the way it should be. But first I need to find a way to accurately identify all the duplicate content that exists out in googleland. Any ideas?

    BTW, the site's url is istreamcom.com. Some reasons why there may be duplicate content are:
    - I had some other 'beta' versions of the site up and where live under subdomain names of the main url (they have been taken down now though)
    - When I moved the site over to a new server I first created a subdomain called 'istreamcom.com.aphonestore.com' and this is where the 'www.istreamcom.com' still currently reside, its just that with the host I use, when I setup an additional domain on my server account, I have to first create a subdomain of the origional domain that I used to open the account with and then configure what they call an 'addon' domain, which basically points to the subdomain (so www.istreamcom.com points to istreamcom.com.aphonestore.com). I am hoping that since istreamcom.com.aphonestore.com, this is going to be an ongoing issue with the duplicate content.
    Last edited by budbiss; Jan 10th, 2005 at 06:42 PM. Reason: Forgot to mention something
  2. #2
  3. No Profile Picture
    Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Oct 2004
    Location
    Maryland
    Posts
    154
    Rep Power
    15
    Copyscape.com is a nice resource for checking dup content.
  4. #3
  5. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2004
    Posts
    72
    Rep Power
    15
    Thanks Sharp, someone recommended copyscape to me earlier and I did run my url with them and 10 results returned, but I have no idea what this means? What exactly do these 10 results signify?

    Originally Posted by Sharp23
    Copyscape.com is a nice resource for checking dup content.
  6. #4
  7. No Profile Picture
    Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Oct 2004
    Location
    Maryland
    Posts
    154
    Rep Power
    15
    Originally Posted by budbiss
    Thanks Sharp, someone recommended copyscape to me earlier and I did run my url with them and 10 results returned, but I have no idea what this means? What exactly do these 10 results signify?
    Did you click on the links for those sites? Were any of them internal pages from your site? I'm pretty sure copyscape will turn up dup content from your own site, but I don't have any sites to test the theory on right now.

    I ran one for a client just now, because of your post. I found some blatant copyright infringement (on someone's part, heh). Now I'm wondering if it was my client or them, wonder if I should tell them.
  8. #5
  9. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2004
    Posts
    72
    Rep Power
    15
    BTW, I just realized there are more versions of my site that are indexed. Together it appears that these are all indexed on SEs:
    rev2.istreamcom.com
    rev3.istreamcom.com
    beta2.istreamcom.com
    istreamcom.com.aphonestore.com

    It seems as though, slowly the 'istreamcom.com.aphonestore.com' links are being replaced by just 'www.istreamcom.com', but as for the other 3 addresses, when will they be dropped from the search listings? I mean, they don't even go anywhere anymore. This is such a mess. I have learned my lesson...robots.txt is a very useful tool ;) Just wish I would have discovered it and used it earlier. Someone recommended setting up a robots.txt that would tell the spider to remove the links it has of its address...should I try this? The only thing is that I deleted all of the files from these subdomains, so if I did try it, I would just be creating new subdomains with those names and putting just the robots.txt in them.
    Last edited by budbiss; Jan 10th, 2005 at 07:09 PM.
  10. #6
  11. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2004
    Posts
    72
    Rep Power
    15
    Originally Posted by Sharp23
    Did you click on the links for those sites? Were any of them internal pages from your site? I'm pretty sure copyscape will turn up dup content from your own site, but I don't have any sites to test the theory on right now.
    It returned 9 results...I copied them to the bottom of this post with comments for each one. BTW, none of them are of 'www.istreamcom.com', all of the six out of nine that are actually ours are listed under 'istreamcom.com.aphonestore.com'.

    Originally Posted by Sharp23
    I ran one for a client just now, because of your post. I found some blatant copyright infringement (on someone's part, heh). Now I'm wondering if it was my client or them, wonder if I should tell them.
    Interesting.



    This is us, but it is using the 'istreamcom.com.aphonestore.com' address:
    iStream Communications New And Refurbished Business Telephone ...
    ... Complete Mitel SX-50 System Regular Price: $4,995.95 Sale Price: $4,575.95, Complete ... 20 System Regular Price: $1,795.00 Sale Price: $1,650.00, Mitel Superset 4150 (Like New) Regular ... Price: $69.00, EZ Call-A-Matic PMS Lite Regular Price: $1,095.00 ... Price: $139.00 Sale Price: $99.00, Mitel Superset 410 (Like New) Regular ...
    http://istreamcom.com.aphonestore.com/catalog/index.php - cached

    This is an SE result:
    LookSmart - Search results for "mitel"
    ... iStream Communications > Mitel Toll ... 1-866-696-4835 Search: Home | About Us | Contact Us | Repairs | Tech Support My Cart | Check Out Browse by Manufacturer Aleen Avaya Cisco DuVoice EZ Advantage ... Mitel Nortel Plantronics ...
    http://64.241.242.202/ r_search?l&key=mitel&skip=135&se=2,0,7,300&search= us302562;local_US - cached

    This is us, but it is using the 'istreamcom.com.aphonestore.com' address:
    iStream Communications > EZ Advantage
    ... iStream Communications, Toll Free: 1-866-696-4835, Search: Home | About Us | Contact Us | Repairs | Tech Support, My Cart My Cart | Check Out. ...
    http://istreamcom.com.aphonestore.com/catalog/index.php?manufacturers_id=15 - cached

    This appears to be the address of our old SSL cert (its nonfunctional)
    iStream Communications New And Refurbished Business Telephone ...
    ... 4000 Regular Price: $229.95 Sale Price: $199.95, Mitel Superset 4025 (Like New) Regular Price: $169.00 Sale Price: $149.00, Mitel Superset 430 (Like New) Regular ...
    https://secure.activeintra.net/istreamcom.com/catalog/ - cached

    This is us, but it is using the 'istreamcom.com.aphonestore.com' address:
    iStream Communications : Mitel Superset 420 (Like New)
    ... Mitel Superset 420 (Like New). Regular Price: $139.00 Sale Price: $99.00. Mitel Superset 420 (Like New) Click to enlarge. Product ...
    http://istreamcom.com.aphonestore.com/catalog/product_info.php?products_id=69 - cached

    This is us, but it is using the 'istreamcom.com.aphonestore.com' address:
    iStream Communications : Mitel Superset 430 (Like New)
    ... Mitel Superset 430 (Like New). Regular Price: $109.00 Sale Price: $99.00. Mitel Superset 430 (Like New) Click to enlarge. Product ...
    http://istreamcom.com.aphonestore.com/catalog/product_info.php?products_id=70 - cached

    This is us, but it is using the 'istreamcom.com.aphonestore.com' address:
    iStream Communications : Mitel Superset 4150 (Like New)
    ... Mitel Superset 4150 (Like New). Regular Price: $249.00 Sale Price: $219.00. Mitel Superset 4150 (Like New) Click to enlarge. ...
    http://istreamcom.com.aphonestore.com/catalog/product_info.php?products_id=241 - cached

    This is us, but it is using the 'istreamcom.com.aphonestore.com' address:
    iStream Communications > Mitel
    ... $1,795.00 $1,650.00, Buy Now. Complete Mitel SX-50 System, Complete Mitel SX-50 System, $4,995.95 $4,575.95, Buy Now. Mitel 9102-004-000 ...
    http://istreamcom.com.aphonestore.com/catalog/index.php?manufacturers_id=10 - cached

    This is a competitors website...not exactly sure why it came up.
    Atlas Systems, Inc. specilizes in new and used/refurbished phone ...
    ... Mitel Superset 4150 Like NEW. Show order | Info | Privacy Policy | Home. ... Mitel Superset 4025 Like NEW. Mitel Superset 4150 Like NEW $189.00, 10/$1,849.00 ...
    http://shop.store.yahoo.com/atlasphones/mi40seph.html - cached
  12. #7
  13. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2004
    Posts
    72
    Rep Power
    15
    Thanks for the advice. I have a few questions though if you don't mind:

    1) How do I address the ones that are mine? I mean, what action do I take to fix them?

    2) As far as the other ones that are showing up that aren't on my server, what exactly are they? How are they considered duplicates? I'm not sure I understand. Why would they be considered to be duplicates if they're not on my server?

    3) You mention to run this for each page. What exactly do you mean by that? Are you saying I need to run each page I have through copyscape?

    4) What benefit to others get by having duplicates of my sites content?

    Originally Posted by gchaney
    HI Budbis,

    You're simply going to have to address these yourself. The ones that come from domains under your control should be addressed with appropriate changes. You'll need to address the third party domains directly with the site owner(s), if that does not work, address with the host service and with the search engines.

    Run this for each of your pages and repeat. Yes, site theft is very expensive, both in time and money unfortunately.

    Good Luck!

    and a hearty Cheers!
  14. #8
  15. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2004
    Posts
    72
    Rep Power
    15
    Originally Posted by budbiss
    - When I moved the site over to a new server I first created a subdomain called 'istreamcom.com.aphonestore.com' and this is where the 'www.istreamcom.com' still currently reside, its just that with the host I use, when I setup an additional domain on my server account, I have to first create a subdomain of the origional domain that I used to open the account with and then configure what they call an 'addon' domain, which basically points to the subdomain (so www.istreamcom.com points to istreamcom.com.aphonestore.com). I am hoping that since istreamcom.com.aphonestore.com, this is going to be an ongoing issue with the duplicate content.
    Also, can someone please lend me their thoughts on this topic as well? Is there some safegaurd I should put into place to prevent spiders from grabbing stuff as 'istreamcom.com.aphonestore.com' instead of 'www.istreamcom.com'? Or is there really nothing I can do about this? Or is there no reason to do anything about this?

    Thanks!
  16. #9
  17. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2004
    Posts
    72
    Rep Power
    15
    Can somebody please help me out and give me their comments on my last two posts on this thread?

IMN logo majestic logo threadwatch logo seochat tools logo