Forum: Search Engine Spiders

Search Engine Spiders forum discussing how to know which search engines are visiting your site by identifying their spiders. Share their IPs and spidering habits, and plan for them to get better SEO results.

Threads 1 to 40 of 529
Page 1 of 14 12311 ... Last
  • Jump to page:
    • Replies: 4
    • Views: 2,205
    Feb 28th, 2013, 06:58 AM
  1. Why Google will store links along with the content ?

    Started by naganna, Feb 15th, 2013 04:59 AM
    Tag List
    • Replies: 0
    • Views: 1,968
    Feb 15th, 2013, 04:59 AM
  2. Nofollow Over Use

    Started by ragecrush, May 15th, 2012 08:45 AM
    • Replies: 1
    • Views: 1,959
    Feb 13th, 2013, 02:58 AM
  3. How do I create a spider-friendly site map?

    Started by riya15, Aug 29th, 2011 06:13 AM
    • Replies: 3
    • Views: 3,465
    Feb 13th, 2013, 01:48 AM
  4. Restricted site and search engine crawling

    Started by ciclonpr, Aug 19th, 2012 09:32 AM
    • Replies: 1
    • Views: 3,029
    Feb 12th, 2013, 01:16 PM
  5. How can I create a robot.txt file using google?

    Started by suprity, Dec 3rd, 2012 08:29 PM
    • Replies: 5
    • Views: 2,540
    Feb 12th, 2013, 01:07 PM
  6. Google Indexing problem Magento Store

    Started by vickydevries, Jul 16th, 2012 04:38 AM
    • Replies: 1
    • Views: 3,138
    Feb 7th, 2013, 01:36 PM
  7. Xml sitemap submit

    Started by Sharon Lisa, Dec 3rd, 2012 03:53 AM
    • Replies: 2
    • Views: 2,407
    Feb 7th, 2013, 01:26 PM
  8. Multiple Sitemaps

    Started by jamesone, Dec 20th, 2012 02:29 PM
    • Replies: 4
    • Views: 2,795
    Jan 1st, 2013, 02:33 AM
  9. Yet another robots.txt question

    Started by Jef Patat, Jan 19th, 2005 09:35 AM
    • Replies: 4
    • Views: 3,001
    Dec 23rd, 2012, 12:53 AM
  10. Question Search engine spider crawling?

    Started by siva_lingam, Sep 28th, 2005 05:14 AM
    • Replies: 3
    • Views: 1,971
    Dec 21st, 2012, 03:47 AM
  11. Forum Tracker

    Started by mchilly, Oct 29th, 2006 11:01 PM
    • Replies: 3
    • Views: 2,313
    Dec 19th, 2012, 03:10 AM
  12. Are Mobile Redirects Harmful to SEO

    Started by thomop, Nov 22nd, 2012 05:04 AM
    • Replies: 4
    • Views: 1,046
    Nov 23rd, 2012, 04:42 PM
  13. Is this Robots.txt for my wordpress Blog SEO Friendly?

    Started by webhush, Apr 28th, 2009 11:39 AM
    2 Pages
    1 2
    • Replies: 21
    • Views: 6,212
    Nov 11th, 2012, 10:36 AM
  14. Indexing went down to 0

    Started by Deepali, Aug 30th, 2012 02:20 AM
    • Replies: 4
    • Views: 1,117
    Oct 29th, 2012, 12:26 PM
  15. Content examination in lynx text base browser

    Started by seodip, Jul 24th, 2012 06:54 AM
    • Replies: 0
    • Views: 1,317
    Jul 24th, 2012, 06:54 AM
  16. Thumbs up Sitemap

    Started by Stela Wilson, May 1st, 2012 07:33 PM
    • Replies: 6
    • Views: 1,927
    Jul 20th, 2012, 01:23 AM
  17. What is search engine spider?

    Started by olloboy, Jul 6th, 2012 11:26 PM
    • Replies: 3
    • Views: 1,385
    Jul 17th, 2012, 06:27 AM
  18. Google Robot

    Started by im.amideb, Jul 11th, 2012 07:52 AM
    • Replies: 2
    • Views: 1,436
    Jul 13th, 2012, 05:54 AM
  19. What is Spiders and How They Effect the website

    Started by ilmkidunya, Jan 17th, 2012 12:01 AM
    • Replies: 4
    • Views: 2,356
    Jun 12th, 2012, 04:38 AM
  20. Use of robot.txt!!

    Started by Seo picker, Apr 4th, 2012 05:35 AM
    • Replies: 3
    • Views: 1,769
    May 29th, 2012, 10:30 AM
  21. What is sitemap?

    Started by robert4u, May 29th, 2012 02:46 AM
    • Replies: 1
    • Views: 1,263
    May 29th, 2012, 03:43 AM
  22. Help desk software

    Started by boomiurandai, Feb 23rd, 2008 07:08 AM
    • Replies: 2
    • Views: 2,627
    May 21st, 2012, 10:11 PM
  23. DoCoMo/1.0/P502i/c10 (Google CHTML Proxy/1.0)

    Started by wirelesscove, May 1st, 2003 04:04 PM
    • Replies: 5
    • Views: 5,992
    May 8th, 2012, 02:17 AM
  24. Pageid=13 - robots.txt help!

    Started by m_grindy, May 3rd, 2012 09:48 AM
    • Replies: 2
    • Views: 1,459
    May 7th, 2012, 08:42 PM
  25. Help! Site disappeared off Cache

    Started by ajithlaxman, Aug 18th, 2010 12:46 AM
    • Replies: 5
    • Views: 2,296
    May 1st, 2012, 12:59 PM
  26. Sitemap.xml

    Started by wgrasty, Feb 13th, 2012 08:08 AM
    • Replies: 6
    • Views: 2,184
    Apr 25th, 2012, 10:43 AM
  27. Lightbulb My Website Ranking Dropping in Google

    Started by Devbhushan, Sep 16th, 2010 05:14 AM
    • Replies: 7
    • Views: 2,924
    Mar 29th, 2012, 06:29 AM
  28. How does Google Crawl Geo Dynamic Content?

    Started by debeenus, Mar 21st, 2012 09:25 AM
    • Replies: 0
    • Views: 1,751
    Mar 21st, 2012, 09:25 AM
  29. Parsing php in .html files & seo

    Started by Sienna, Feb 29th, 2012 12:13 AM
    • Replies: 1
    • Views: 1,927
    Feb 29th, 2012, 01:33 AM
  30. Regional access restrictions and search indexing

    Started by Guest123, Jan 31st, 2012 06:08 AM
    • Replies: 1
    • Views: 1,507
    Feb 3rd, 2012, 10:16 AM
  31. Smile Seo strategies

    Started by nitish123, Dec 24th, 2011 03:33 AM
    • Replies: 2
    • Views: 1,816
    Jan 25th, 2012, 08:37 AM
  32. How to get googlebot to visit

    Started by cnile, Dec 29th, 2003 12:33 PM
    2 Pages
    1 2
    • Replies: 24
    • Views: 30,618
    Jan 18th, 2012, 04:49 AM
  33. 2 questions...driving me crazy

    Started by mechele7, Dec 26th, 2011 08:51 AM
    • Replies: 0
    • Views: 1,585
    Dec 26th, 2011, 08:51 AM
  34. Googlebot requesting googlelongnumber.html ??

    Started by Savage, Nov 8th, 2011 12:53 PM
    • Replies: 3
    • Views: 1,992
    Nov 9th, 2011, 01:04 PM
  35. TurnitinBot

    Started by st, Aug 12th, 2003 08:08 AM
    • Replies: 4
    • Views: 4,740
    Nov 2nd, 2011, 02:22 AM
  36. Crawling problem

    Started by stunning boy, Sep 3rd, 2011 05:26 AM
    • Replies: 3
    • Views: 2,344
    Oct 28th, 2011, 06:25 PM
  37. Question How to stop search options from being indexed

    Started by dan1el, Mar 11th, 2011 03:43 AM
    • Replies: 5
    • Views: 2,019
    Oct 19th, 2011, 03:49 PM
  38. Can a php path be indexed?

    Started by michaelw, Aug 9th, 2011 12:50 PM
    • Replies: 1
    • Views: 1,704
    Sep 15th, 2011, 01:19 AM
  39. Problem with robots on dynamic page.

    Started by dan32, May 2nd, 2011 06:46 PM
    • Replies: 4
    • Views: 2,534
    Sep 8th, 2011, 05:24 PM
Page 1 of 14 12311 ... Last
  • Jump to page:

Forum Information and Options

Users Browsing this Forum

There are currently 1 users browsing this forum. (0 members & 1 guests)

Thread Display Options

Use this control to limit the display of threads to those newer than the specified time frame.

Allows you to choose the data by which the thread list will be sorted.

Order threads in...

Note: when sorting by date, 'descending order' will show the newest results first.


IMN logo majestic logo threadwatch logo seochat tools logo