Forum: Search Engine Spiders

Search Engine Spiders forum discussing how to know which search engines are visiting your site by identifying their spiders. Share their IPs and spidering habits, and plan for them to get better SEO results.

Threads 1 to 40 of 529
Page 1 of 14 12311 ... Last
  • Jump to page:
    • Replies: 4
    • Views: 1,965
    Feb 28th, 2013, 06:58 AM
  1. Why Google will store links along with the content ?

    Started by naganna, Feb 15th, 2013 04:59 AM
    Tag List
    • Replies: 0
    • Views: 1,739
    Feb 15th, 2013, 04:59 AM
  2. Nofollow Over Use

    Started by ragecrush, May 15th, 2012 08:45 AM
    • Replies: 1
    • Views: 1,727
    Feb 13th, 2013, 02:58 AM
  3. How do I create a spider-friendly site map?

    Started by riya15, Aug 29th, 2011 06:13 AM
    • Replies: 3
    • Views: 3,228
    Feb 13th, 2013, 01:48 AM
  4. Restricted site and search engine crawling

    Started by ciclonpr, Aug 19th, 2012 09:32 AM
    • Replies: 1
    • Views: 2,691
    Feb 12th, 2013, 01:16 PM
  5. How can I create a robot.txt file using google?

    Started by suprity, Dec 3rd, 2012 08:29 PM
    • Replies: 5
    • Views: 2,172
    Feb 12th, 2013, 01:07 PM
  6. Google Indexing problem Magento Store

    Started by vickydevries, Jul 16th, 2012 04:38 AM
    • Replies: 1
    • Views: 2,593
    Feb 7th, 2013, 01:36 PM
  7. Xml sitemap submit

    Started by Sharon Lisa, Dec 3rd, 2012 03:53 AM
    • Replies: 2
    • Views: 2,060
    Feb 7th, 2013, 01:26 PM
  8. Multiple Sitemaps

    Started by jamesone, Dec 20th, 2012 02:29 PM
    • Replies: 4
    • Views: 2,434
    Jan 1st, 2013, 02:33 AM
  9. Yet another robots.txt question

    Started by Jef Patat, Jan 19th, 2005 09:35 AM
    • Replies: 4
    • Views: 2,753
    Dec 23rd, 2012, 12:53 AM
  10. Question Search engine spider crawling?

    Started by siva_lingam, Sep 28th, 2005 05:14 AM
    • Replies: 3
    • Views: 1,707
    Dec 21st, 2012, 03:47 AM
  11. Forum Tracker

    Started by mchilly, Oct 29th, 2006 11:01 PM
    • Replies: 3
    • Views: 2,097
    Dec 19th, 2012, 03:10 AM
  12. Are Mobile Redirects Harmful to SEO

    Started by thomop, Nov 22nd, 2012 05:04 AM
    • Replies: 4
    • Views: 769
    Nov 23rd, 2012, 04:42 PM
  13. Is this Robots.txt for my wordpress Blog SEO Friendly?

    Started by webhush, Apr 28th, 2009 11:39 AM
    2 Pages
    1 2
    • Replies: 21
    • Views: 5,386
    Nov 11th, 2012, 10:36 AM
  14. Indexing went down to 0

    Started by Deepali, Aug 30th, 2012 02:20 AM
    • Replies: 4
    • Views: 915
    Oct 29th, 2012, 12:26 PM
  15. Content examination in lynx text base browser

    Started by seodip, Jul 24th, 2012 06:54 AM
    • Replies: 0
    • Views: 1,056
    Jul 24th, 2012, 06:54 AM
  16. Thumbs up Sitemap

    Started by Stela Wilson, May 1st, 2012 07:33 PM
    • Replies: 6
    • Views: 1,562
    Jul 20th, 2012, 01:23 AM
  17. What is search engine spider?

    Started by olloboy, Jul 6th, 2012 11:26 PM
    • Replies: 3
    • Views: 1,184
    Jul 17th, 2012, 06:27 AM
  18. Google Robot

    Started by im.amideb, Jul 11th, 2012 07:52 AM
    • Replies: 2
    • Views: 1,071
    Jul 13th, 2012, 05:54 AM
  19. What is Spiders and How They Effect the website

    Started by ilmkidunya, Jan 17th, 2012 12:01 AM
    • Replies: 4
    • Views: 2,041
    Jun 12th, 2012, 04:38 AM
  20. Use of robot.txt!!

    Started by Seo picker, Apr 4th, 2012 05:35 AM
    • Replies: 3
    • Views: 1,481
    May 29th, 2012, 10:30 AM
  21. What is sitemap?

    Started by robert4u, May 29th, 2012 02:46 AM
    • Replies: 1
    • Views: 1,027
    May 29th, 2012, 03:43 AM
  22. Help desk software

    Started by boomiurandai, Feb 23rd, 2008 07:08 AM
    • Replies: 2
    • Views: 2,319
    May 21st, 2012, 10:11 PM
  23. DoCoMo/1.0/P502i/c10 (Google CHTML Proxy/1.0)

    Started by wirelesscove, May 1st, 2003 04:04 PM
    • Replies: 5
    • Views: 5,680
    May 8th, 2012, 02:17 AM
  24. Pageid=13 - robots.txt help!

    Started by m_grindy, May 3rd, 2012 09:48 AM
    • Replies: 2
    • Views: 1,174
    May 7th, 2012, 08:42 PM
  25. Help! Site disappeared off Cache

    Started by ajithlaxman, Aug 18th, 2010 12:46 AM
    • Replies: 5
    • Views: 1,949
    May 1st, 2012, 12:59 PM
  26. Sitemap.xml

    Started by wgrasty, Feb 13th, 2012 08:08 AM
    • Replies: 6
    • Views: 1,792
    Apr 25th, 2012, 10:43 AM
  27. Lightbulb My Website Ranking Dropping in Google

    Started by Devbhushan, Sep 16th, 2010 05:14 AM
    • Replies: 7
    • Views: 2,617
    Mar 29th, 2012, 06:29 AM
  28. How does Google Crawl Geo Dynamic Content?

    Started by debeenus, Mar 21st, 2012 09:25 AM
    • Replies: 0
    • Views: 1,386
    Mar 21st, 2012, 09:25 AM
  29. Parsing php in .html files & seo

    Started by Sienna, Feb 29th, 2012 12:13 AM
    • Replies: 1
    • Views: 1,658
    Feb 29th, 2012, 01:33 AM
  30. Regional access restrictions and search indexing

    Started by Guest123, Jan 31st, 2012 06:08 AM
    • Replies: 1
    • Views: 1,296
    Feb 3rd, 2012, 10:16 AM
  31. Smile Seo strategies

    Started by nitish123, Dec 24th, 2011 03:33 AM
    • Replies: 2
    • Views: 1,634
    Jan 25th, 2012, 08:37 AM
  32. How to get googlebot to visit

    Started by cnile, Dec 29th, 2003 12:33 PM
    2 Pages
    1 2
    • Replies: 24
    • Views: 29,424
    Jan 18th, 2012, 04:49 AM
  33. 2 questions...driving me crazy

    Started by mechele7, Dec 26th, 2011 08:51 AM
    • Replies: 0
    • Views: 1,400
    Dec 26th, 2011, 08:51 AM
  34. Googlebot requesting googlelongnumber.html ??

    Started by Savage, Nov 8th, 2011 12:53 PM
    • Replies: 3
    • Views: 1,446
    Nov 9th, 2011, 01:04 PM
  35. TurnitinBot

    Started by st, Aug 12th, 2003 08:08 AM
    • Replies: 4
    • Views: 4,275
    Nov 2nd, 2011, 02:22 AM
  36. Crawling problem

    Started by stunning boy, Sep 3rd, 2011 05:26 AM
    • Replies: 3
    • Views: 1,996
    Oct 28th, 2011, 06:25 PM
  37. Question How to stop search options from being indexed

    Started by dan1el, Mar 11th, 2011 03:43 AM
    • Replies: 5
    • Views: 1,811
    Oct 19th, 2011, 03:49 PM
  38. Can a php path be indexed?

    Started by michaelw, Aug 9th, 2011 12:50 PM
    • Replies: 1
    • Views: 1,509
    Sep 15th, 2011, 01:19 AM
  39. Problem with robots on dynamic page.

    Started by dan32, May 2nd, 2011 06:46 PM
    • Replies: 4
    • Views: 2,289
    Sep 8th, 2011, 05:24 PM
Page 1 of 14 12311 ... Last
  • Jump to page:

Forum Information and Options

Users Browsing this Forum

There are currently 1 users browsing this forum. (0 members & 1 guests)

Thread Display Options

Use this control to limit the display of threads to those newer than the specified time frame.

Allows you to choose the data by which the thread list will be sorted.

Order threads in...

Note: when sorting by date, 'descending order' will show the newest results first.


IMN logo majestic logo threadwatch logo seochat tools logo