#1
  1. No Profile Picture
    Contributing User
    SEO Chat Good Citizen (1000 - 1499 posts)

    Join Date
    Jun 2006
    Location
    England
    Posts
    1,057
    Rep Power
    159

    Forcing The Spider To Go Deep


    Im in the middle of a large clean up (301) of : Old, Thin or Trashy Articles on my site. Same date back a lot of years and probably the spider doesnt see them very often.

    Is there a way to force the spider to take a deep look at my site, or is it just a matter of waiting for it to eventually take another look for these articles.
    All The Fun Of The Fair !!!
  2. #2
  3. SEO Consultant
    SEO Chat High Scholar (3500 - 3999 posts)

    Join Date
    Apr 2010
    Location
    Atlanta, GA
    Posts
    3,587
    Rep Power
    1944
    A sitemap would help.

    Creating a strong menu structure would help, too.

    Are you using a CMS? They have pretty good archiving and structuring options.

    Comments on this post

    • EGOL agrees
    www.V2INTERACTIVE.net - SEO Web Design | On-Page SEO | SEO Audits & Reports | Link Building
    BASED IN ATLANTA, GA. Contact: Skype/Phone/Email (Discounted rates available for SEOChat members only - email seochat@v2interactive.net)
    Please EMAIL me at the above address and do not send Private Messages. I rarely check them and your message will be lost.
  4. #3
  5. Agency SEO
    SEO Chat Mastermind (5000+ posts)

    Join Date
    Aug 2008
    Location
    Florida
    Posts
    6,113
    Rep Power
    2826
    The amount of pages of your site that Google will index is related to your PR. Build quality inbound links to your interior pages and you will get them all crawled

    Comments on this post

    • EGOL agrees
  6. #4
  7. No Profile Picture
    Contributing User
    SEO Chat Good Citizen (1000 - 1499 posts)

    Join Date
    Jun 2006
    Location
    England
    Posts
    1,057
    Rep Power
    159
    Cheers guys, I think the majority do get crawled, its just the site is quite large and and some of the posts havent changed over the years so im certain the spider wont go to some of them very often at all, and certainly the most important ones do get crwled. Sadly I had a lot of poor quality pages. Some were news related that are no longer relevent some were thin pages with little quality. Im deleting them and redirecting to category homepage or other relevent page with better content.

    And yes Im running cms, well its wordpress, I do have yoast sitemap plugin, but Ive never been sure if it actually works. I really need to learn how to get the best out of it. One step at a time though as Im concentrating on getting the rubbish off the site. Im aiming at fewer higher quality pages.
  8. #5
  9. No Profile Picture
    Contributing User
    SEO Chat Good Citizen (1000 - 1499 posts)

    Join Date
    Apr 2009
    Posts
    1,283
    Rep Power
    514
    Generally speaking the more changes Googlebot sees the more it will look. Which makes sense because if you completely re-did your site or the domain was sold it would need a fresh index of everything. If you are talking mostly 301s it might not care because it doesn't effect users much. If a lot of pages actually changed, especially completely, google will tend to start poking around to see what else changed.

    You can submit a site map but in the end you are mostly looking at waiting for Google to do it's thing.

Similar Threads

  1. Advantages to forcing WWW with .htaccess
    By warehousemultim in forum Google Optimization
    Replies: 1
    Last Post: Mar 19th, 2009, 01:34 PM
  2. Replies: 7
    Last Post: Dec 3rd, 2004, 06:14 PM
  3. Spider Alert Script - emails you when a spider visits
    By ryanturner.com in forum Web Design, Coding and Programming
    Replies: 0
    Last Post: Dec 1st, 2004, 12:46 PM
  4. Any tips for making Yahoo Spider go more than one level deep on a dynamic site?
    By Lionzshare in forum BING/Yahoo Search Optimization
    Replies: 4
    Last Post: Oct 28th, 2004, 01:58 PM

IMN logo majestic logo threadwatch logo seochat tools logo