#1
  1. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Jan 2011
    Location
    Buffalo, NY
    Posts
    315
    Rep Power
    539

    Nofollow Links Get Found but are They Crawled?


    I've read others mentioning that Google will find nofollow links but not pass page rank. Do they still crawl the page or just take note that it has links. I ask because Google mentions using nofollow for crawl prioritization but that doesn't make sense if they are still crawling the page anyway.

    https://support.google.com/webmaster...er/96569?hl=en

    If I have ecommerce filters that have nofollow applied, I understand that Google won't pass pagerank to them yet they are aware the page exists. Are they still crawling that page? Is that using crawl budget?
  2. #2
  3. Contributing User
    SEO Chat Adventurer (500 - 999 posts)

    Join Date
    Apr 2011
    Posts
    967
    Rep Power
    1550
    I've tried this, Google crawls them anyway!
  4. #3
  5. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Jan 2011
    Location
    Buffalo, NY
    Posts
    315
    Rep Power
    539
    Interesting, I wonder why Google even mentions the crawl prioritization then.

    The only other thing I can think that would stop Google from crawling is, if the filter uses a url parameter you could tell Google what it does and not to crawl but, that is just a suggestion. And not helpful if your filters use "seo-friendly" urls without parameters.

    (Robots.txt would stop them too but, that doesn't seem like a good idea.)
  6. #4
  7. SEO Consultant
    SEO Chat Genius (4000 - 4499 posts)

    Join Date
    Jul 2004
    Location
    Minneapolis, MN, USA
    Posts
    4,244
    Rep Power
    1106
    Having a nofollow link to (TO is important) a URL doesn't mean that URL won't get indexed. Merely visiting a page with a web browser can get a page indexed (by a number of ways). The nofollow designation TO a link FROM a page just says I don't rate that URL as important as others that I didn't designate as nofollow, therefore it is prioritized lower. A lower priority doesn't mean zero chance of being indexed, just less than others.

    Also, if there are other links both internal and external to a URL that either do or don't have a nofollow link there is also another path and any one particular URLs nofollow link to another URL does not control how that URL is indexed.

    Comments on this post

    • ryandiscord agrees : Great explanation!
  8. #5
  9. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Jan 2011
    Location
    Buffalo, NY
    Posts
    315
    Rep Power
    539
    Applying that to my filters situation, I would be totally okay with a link or two pointing to a filter variation. Most cases I have a canonical tag pointing to the unfiltered view anyway.

    Where I get concerned is that Google could spend a lot of extra time crawling every url variation created by these filters, if it were to crawl everything including nofollow on that page.
  10. #6
  11. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    May 2016
    Posts
    14
    Rep Power
    0
    Gosh, I was going to start a thread for this. So inherently nofollow links command to crawler not to crawl and index the page. but then what is the use of a such a meta tag as following one. ? If nofollow attribute says not to index as well why use noindex specifically ?

    <meta name="robot" content="nofollow,noindex" />
  12. #7
  13. SEO Consultant
    SEO Chat Genius (4000 - 4499 posts)

    Join Date
    Jul 2004
    Location
    Minneapolis, MN, USA
    Posts
    4,244
    Rep Power
    1106
    There is a difference between adding the nofollow and noindex definitions in a page, rather than a link TO a page.
    The tag that you show there should prevent the page from being indexed by all compliant spiders.
    Just putting a nofollow in a link to a page will not stop the linked page getting indexed.
  14. #8
  15. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    May 2016
    Posts
    14
    Rep Power
    0
    Originally Posted by tstolber
    There is a difference between adding the nofollow and noindex definitions in a page, rather than a link TO a page.
    The tag that you show there should prevent the page from being indexed by all compliant spiders.
    Just putting a nofollow in a link to a page will not stop the linked page getting indexed.
    Got it mate, That makes sense. I must have been drunk or something. Thank you!
    Last edited by joshbrendon; Sep 15th, 2016 at 04:40 AM.
  16. #9
  17. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Sep 2016
    Posts
    7
    Rep Power
    0
    Yes no follow links get crawled by Google Spiders or bots. The reason why they are important is to balance the number do follow and no follow links. No follow also helps in increasing the websites traffic.
  18. #10
  19. Super Moderator
    SEO Chat Mastermind (5000+ posts)

    Join Date
    Mar 2004
    Location
    Gloucester (South West UK).
    Posts
    6,533
    Rep Power
    3520
    Originally Posted by tstolber
    The nofollow designation TO a link FROM a page just says I don't rate that URL as important as others that I didn't designate as nofollow, therefore it is prioritized lower.
    My understanding of the "nofollow" tag is that it prevents the passage of "page-rank/link-juice" to the linked page; (in effect it "disavows" the link).
    The link is still crawled (and soaks it's share of "link-juice/page-rank" away from the page), so the linked url will be 'discovered' as normal.
    As for the page-rank/link-juice... it's gone
    Last edited by ClickyB; Sep 22nd, 2016 at 12:21 PM.
    ClickyB
    "The quality of the visitor is more important than the volume..." (Egol 22nd Feb 2008)
    [New to SEO/SeoChat?] [Canonical Problems?] [Forum Rules & Posting Guidelines]
  20. #11
  21. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Jan 2011
    Location
    Buffalo, NY
    Posts
    315
    Rep Power
    539
    So basically, nofollow on filter links is probably not the way to go. I have looked at other big ecommerce sites, some I have seen use a /filter/ directory and block in robots.txt, some use form elements and others just use regular links with canonical tags on the filters.

    Robots definitely controls crawl budget but doesn't stop the pagerank from dispersing, also if someone links to a filter instead of the main page it would never get found.

    ClickyB, what are your thoughts on how pagerank is passed from a link to a page that has a canonical tag to the page with the original link? Example) Shoes Page, with a link to a filtered version of Green Shoes that has a canonical tag back to Shoes. Does that pagerank circle back?
  22. #12
  23. Super Moderator
    SEO Chat Mastermind (5000+ posts)

    Join Date
    Mar 2004
    Location
    Gloucester (South West UK).
    Posts
    6,533
    Rep Power
    3520
    Originally Posted by ryandiscord
    ClickyB, what are your thoughts on how pagerank is passed from a link to a page that has a canonical tag to the page with the original link? Example) Shoes Page, with a link to a filtered version of Green Shoes that has a canonical tag back to Shoes. Does that pagerank circle back?
    According to Moz (here):
    Originally Posted by moz
    Does Rel=Canonical Pass Authority/PageRank?
    This is very difficult to measure, but if you use rel=canonical appropriately, and if Google honors it, then it appears to act similarly to a 301-redirect. We suspect it passes authority/PageRank for links to the non-canonical URL, with some small amount of loss (similar to a 301).
    That's a "yes"!

    Also according to Google (here):
    Originally Posted by google
    Now, you can simply add this (canonical) <link> tag to specify your preferred version inside the <head> section of the duplicate content URLs:
    Google will understand that the duplicates all refer to the canonical URL.
    Additional URL properties, like PageRank and related signals, are transferred as well.
    Also a "yes"!

    I've never had reason to check/test this myself, but I'm inclined to think the answer is yes.
  24. #13
  25. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Jan 2011
    Location
    Buffalo, NY
    Posts
    315
    Rep Power
    539
    Originally Posted by ClickyB
    I've never had reason to check/test this myself, but I'm inclined to think the answer is yes.
    Cool, thank you! This leads me back to just letting Google crawl the links and using canonical tags appropriately.

Similar Threads

  1. The differences between nofollow links and dofollow links
    By multisystem123 in forum New User SEO Questions and Answers
    Replies: 2
    Last Post: Feb 23rd, 2016, 07:28 AM
  2. NoFollow External links or Internal links?
    By phattrance in forum Google Optimization
    Replies: 10
    Last Post: Jan 2nd, 2009, 12:08 AM
  3. Flash links getting crawled?
    By Marcel in forum Google Optimization
    Replies: 6
    Last Post: Oct 4th, 2003, 03:58 AM
  4. Are banner links crawled by search engines?
    By Michelle in forum Google Optimization
    Replies: 4
    Last Post: Mar 12th, 2003, 04:16 AM

IMN logo majestic logo threadwatch logo seochat tools logo