Forum: Search Engine Spiders

Search Engine Spiders forum discussing how to know which search engines are visiting your site by identifying their spiders. Share their IPs and spidering habits, and plan for them to get better SEO results.

Threads 1 to 40 of 529
Page 1 of 14 12311 ... Last
  • Jump to page:
    • Replies: 4
    • Views: 1,997
    Feb 28th, 2013, 07:58 AM
  1. Why Google will store links along with the content ?

    Started by naganna, Feb 15th, 2013 05:59 AM
    Tag List
    • Replies: 0
    • Views: 1,771
    Feb 15th, 2013, 05:59 AM
  2. Nofollow Over Use

    Started by ragecrush, May 15th, 2012 09:45 AM
    • Replies: 1
    • Views: 1,755
    Feb 13th, 2013, 03:58 AM
  3. How do I create a spider-friendly site map?

    Started by riya15, Aug 29th, 2011 07:13 AM
    • Replies: 3
    • Views: 3,264
    Feb 13th, 2013, 02:48 AM
  4. Restricted site and search engine crawling

    Started by ciclonpr, Aug 19th, 2012 10:32 AM
    • Replies: 1
    • Views: 2,735
    Feb 12th, 2013, 02:16 PM
  5. How can I create a robot.txt file using google?

    Started by suprity, Dec 3rd, 2012 09:29 PM
    • Replies: 5
    • Views: 2,229
    Feb 12th, 2013, 02:07 PM
  6. Google Indexing problem Magento Store

    Started by vickydevries, Jul 16th, 2012 05:38 AM
    • Replies: 1
    • Views: 2,672
    Feb 7th, 2013, 02:36 PM
  7. Xml sitemap submit

    Started by Sharon Lisa, Dec 3rd, 2012 04:53 AM
    • Replies: 2
    • Views: 2,118
    Feb 7th, 2013, 02:26 PM
  8. Multiple Sitemaps

    Started by jamesone, Dec 20th, 2012 03:29 PM
    • Replies: 4
    • Views: 2,476
    Jan 1st, 2013, 03:33 AM
  9. Yet another robots.txt question

    Started by Jef Patat, Jan 19th, 2005 10:35 AM
    • Replies: 4
    • Views: 2,794
    Dec 23rd, 2012, 01:53 AM
  10. Question Search engine spider crawling?

    Started by siva_lingam, Sep 28th, 2005 06:14 AM
    • Replies: 3
    • Views: 1,743
    Dec 21st, 2012, 04:47 AM
  11. Forum Tracker

    Started by mchilly, Oct 30th, 2006 12:01 AM
    • Replies: 3
    • Views: 2,120
    Dec 19th, 2012, 04:10 AM
  12. Are Mobile Redirects Harmful to SEO

    Started by thomop, Nov 22nd, 2012 06:04 AM
    • Replies: 4
    • Views: 812
    Nov 23rd, 2012, 05:42 PM
  13. Is this Robots.txt for my wordpress Blog SEO Friendly?

    Started by webhush, Apr 28th, 2009 12:39 PM
    2 Pages
    1 2
    • Replies: 21
    • Views: 5,534
    Nov 11th, 2012, 11:36 AM
  14. Indexing went down to 0

    Started by Deepali, Aug 30th, 2012 03:20 AM
    • Replies: 4
    • Views: 942
    Oct 29th, 2012, 01:26 PM
  15. Content examination in lynx text base browser

    Started by seodip, Jul 24th, 2012 07:54 AM
    • Replies: 0
    • Views: 1,079
    Jul 24th, 2012, 07:54 AM
  16. Thumbs up Sitemap

    Started by Stela Wilson, May 1st, 2012 08:33 PM
    • Replies: 6
    • Views: 1,589
    Jul 20th, 2012, 02:23 AM
  17. What is search engine spider?

    Started by olloboy, Jul 7th, 2012 12:26 AM
    • Replies: 3
    • Views: 1,208
    Jul 17th, 2012, 07:27 AM
  18. Google Robot

    Started by im.amideb, Jul 11th, 2012 08:52 AM
    • Replies: 2
    • Views: 1,107
    Jul 13th, 2012, 06:54 AM
  19. What is Spiders and How They Effect the website

    Started by ilmkidunya, Jan 17th, 2012 01:01 AM
    • Replies: 4
    • Views: 2,064
    Jun 12th, 2012, 05:38 AM
  20. Use of robot.txt!!

    Started by Seo picker, Apr 4th, 2012 06:35 AM
    • Replies: 3
    • Views: 1,523
    May 29th, 2012, 11:30 AM
  21. What is sitemap?

    Started by robert4u, May 29th, 2012 03:46 AM
    • Replies: 1
    • Views: 1,053
    May 29th, 2012, 04:43 AM
  22. Help desk software

    Started by boomiurandai, Feb 23rd, 2008 08:08 AM
    • Replies: 2
    • Views: 2,343
    May 21st, 2012, 11:11 PM
  23. DoCoMo/1.0/P502i/c10 (Google CHTML Proxy/1.0)

    Started by wirelesscove, May 1st, 2003 05:04 PM
    • Replies: 5
    • Views: 5,737
    May 8th, 2012, 03:17 AM
  24. Pageid=13 - robots.txt help!

    Started by m_grindy, May 3rd, 2012 10:48 AM
    • Replies: 2
    • Views: 1,211
    May 7th, 2012, 09:42 PM
  25. Help! Site disappeared off Cache

    Started by ajithlaxman, Aug 18th, 2010 01:46 AM
    • Replies: 5
    • Views: 1,973
    May 1st, 2012, 01:59 PM
  26. Sitemap.xml

    Started by wgrasty, Feb 13th, 2012 09:08 AM
    • Replies: 6
    • Views: 1,822
    Apr 25th, 2012, 11:43 AM
  27. Lightbulb My Website Ranking Dropping in Google

    Started by Devbhushan, Sep 16th, 2010 06:14 AM
    • Replies: 7
    • Views: 2,644
    Mar 29th, 2012, 07:29 AM
  28. How does Google Crawl Geo Dynamic Content?

    Started by debeenus, Mar 21st, 2012 10:25 AM
    • Replies: 0
    • Views: 1,415
    Mar 21st, 2012, 10:25 AM
  29. Parsing php in .html files & seo

    Started by Sienna, Feb 29th, 2012 01:13 AM
    • Replies: 1
    • Views: 1,690
    Feb 29th, 2012, 02:33 AM
  30. Regional access restrictions and search indexing

    Started by Guest123, Jan 31st, 2012 07:08 AM
    • Replies: 1
    • Views: 1,324
    Feb 3rd, 2012, 11:16 AM
  31. Smile Seo strategies

    Started by nitish123, Dec 24th, 2011 04:33 AM
    • Replies: 2
    • Views: 1,651
    Jan 25th, 2012, 09:37 AM
  32. How to get googlebot to visit

    Started by cnile, Dec 29th, 2003 01:33 PM
    2 Pages
    1 2
    • Replies: 24
    • Views: 30,102
    Jan 18th, 2012, 05:49 AM
  33. 2 questions...driving me crazy

    Started by mechele7, Dec 26th, 2011 09:51 AM
    • Replies: 0
    • Views: 1,418
    Dec 26th, 2011, 09:51 AM
  34. Googlebot requesting googlelongnumber.html ??

    Started by Savage, Nov 8th, 2011 01:53 PM
    • Replies: 3
    • Views: 1,502
    Nov 9th, 2011, 02:04 PM
  35. TurnitinBot

    Started by st, Aug 12th, 2003 09:08 AM
    • Replies: 4
    • Views: 4,308
    Nov 2nd, 2011, 03:22 AM
  36. Crawling problem

    Started by stunning boy, Sep 3rd, 2011 06:26 AM
    • Replies: 3
    • Views: 2,020
    Oct 28th, 2011, 07:25 PM
  37. Question How to stop search options from being indexed

    Started by dan1el, Mar 11th, 2011 04:43 AM
    • Replies: 5
    • Views: 1,831
    Oct 19th, 2011, 04:49 PM
  38. Can a php path be indexed?

    Started by michaelw, Aug 9th, 2011 01:50 PM
    • Replies: 1
    • Views: 1,526
    Sep 15th, 2011, 02:19 AM
  39. Problem with robots on dynamic page.

    Started by dan32, May 2nd, 2011 07:46 PM
    • Replies: 4
    • Views: 2,317
    Sep 8th, 2011, 06:24 PM
Page 1 of 14 12311 ... Last
  • Jump to page:

Forum Information and Options

Users Browsing this Forum

There are currently 1 users browsing this forum. (0 members & 1 guests)

Thread Display Options

Use this control to limit the display of threads to those newer than the specified time frame.

Allows you to choose the data by which the thread list will be sorted.

Order threads in...

Note: when sorting by date, 'descending order' will show the newest results first.


IMN logo majestic logo threadwatch logo seochat tools logo