#1
  1. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Nov 2003
    Posts
    2
    Rep Power
    0

    Google and PHP database crawling


    I did some searching on the board and did not find enough info to answer my question. If the thread exists I apologize now and if you can just point me there that would be great.

    I have a MySQL/PHP webstore. Will google run through all the dynamic links to the products in the database? I searched my logs and only saw a few hits on the hundreds (300+) of possible links.

    www.importpoweronline.com

    Thanks!
  2. #2
  3. No Profile Picture
    Contributing User
    SEO Chat Adventurer (500 - 999 posts)

    Join Date
    Oct 2003
    Posts
    532
    Rep Power
    16

    Re: Google and PHP database crawling


    Originally posted by imp pwr online
    I did some searching on the board and did not find enough info to answer my question. If the thread exists I apologize now and if you can just point me there that would be great.

    I have a MySQL/PHP webstore. Will google run through all the dynamic links to the products in the database? I searched my logs and only saw a few hits on the hundreds (300+) of possible links.

    www.importpoweronline.com

    Thanks!
    Hello,

    Google will be happy to index your data base, but you may not serve google a session id. A session id = A spider trap. If googlebot is even suspecting a spider trap it will stop spidering, better to be safe than sorry.

    I have to suggestions.

    1. Do not pass the session id as part of the url. There are much smarter ways to handel this with php.

    2. Rewrite your urls so they look static. For each end every site I have done this the SERP results have improved.

    Eg.

    Make:
    http://www.importpoweronline.com/cat...roducts_id=277
    Look like:
    http://www.importpoweronline.com/Pro...35_42/227.html


    The best of luck to you.

    PS! You should be cleaning up your markup, it is way below production standard.
  4. #3
  5. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Nov 2003
    Posts
    2
    Rep Power
    0
    I actually did not write the code. It's Os Commerce Open source software.
  6. #4
  7. No Profile Picture
    Contributing User
    SEO Chat Adventurer (500 - 999 posts)

    Join Date
    Oct 2003
    Posts
    532
    Rep Power
    16
    Originally posted by imp pwr online
    I actually did not write the code. It's Os Commerce Open source software.
    Well, you could hire a php programmer to modify the code, session id is the major problem, maybe there already is a fix at the OS commerce website? Remember that any changes you make to open source software will also be opensource.

    The best of luck.
  8. #5
  9. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Nov 2003
    Posts
    12
    Rep Power
    0
    1. Do not pass the session id as part of the url. There are much smarter ways to handel this with php.
    I'd like to compare notes on such experiences.

    This is how I implemented it.

    I looked through the logs looking for bots, then I add them to the global php include file, in which the signatures are in an array.

    Sessions are on by default.
    When a client opens any of the pages on the site, I compare the visitor to the array and if there is a match, I turn off sessions
    because my expect bot has arrived.

    The only slight problem is, what about the bots I haven't included.
    I eliminated the passing of session ID's in the URL's and enabled sessions transmitted in the headers (I think it's handled by cookies that expire when the browser closes), I'm not sure how effective this is.

    The only question I have is, I'd like to remove the bot checking and find a more dynamic way to resolve this.
    Any ideas?

    How would you do something similar?
  10. #6
  11. Contributing User
    SEO Chat Genius (4000 - 4499 posts)

    Join Date
    May 2003
    Location
    Brighton
    Posts
    4,103
    Rep Power
    23
    Google will be happy to index your data base, but you may not serve google a session id. A session id = A spider trap.
    Not entirely true. Google has problems with CERTAIN session ids. I pass ids but only session cookie or have the server rewrite the URL if no cookies are enabled.. You can go here and find out which sessions ids work with Google.

    http://www.hotshotsdesigns.co.uk/google_tests/

    Your URLS are way too dynamic. There are too many params added to the URL along with the session id. Try to keep the URL to one "?" one "&" and one"=". Anything more Google will have a fit about. Look into mod_rewrite to avoid having to reprogram the store and just have the server do it for you. Much more efficient.

    http://www.engelschall.com/pw/apache/rewriteguide/

    Hope that helps and welcome to the board.

    -PK
  12. #7
  13. Head Honcho
    SEO Chat Adventurer (500 - 999 posts)

    Join Date
    Jul 2003
    Location
    Chicago, IL
    Posts
    556
    Rep Power
    16
    Mod_rewrite is the key to the City of Gold.

    There are a few programmers who can accomplish this task for less than $50 - $100.
  14. #8
  15. No Profile Picture
    Xia
    Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Nov 2003
    Posts
    206
    Rep Power
    16
    or you can do it yourself for free if you do a little bit of reading..
  16. #9
  17. Head Honcho
    SEO Chat Adventurer (500 - 999 posts)

    Join Date
    Jul 2003
    Location
    Chicago, IL
    Posts
    556
    Rep Power
    16
    Ugh god that reading is horrible.

    Endershall is a genius, but damn if that **** didn't put me to sleep.

    ;-)
    Take care,

    ... Christopher @ BeRomeo
    -----------------------------------------
    Download your Free Seduction eBook:
    http://www.BeRomeo.com
  18. #10
  19. web designer
    SEO Chat Super Hero (2500 - 2999 posts)

    Join Date
    Aug 2003
    Location
    designing a web site in columbus ohio
    Posts
    2,997
    Rep Power
    50
    while you are making changes....
    why dont you lose that worthless entry page.
    there is no content there at all for google chew on.
    i would just use that second page as the home page.
    you got no meta-desc or meta-keywords going on either.
    some basic seo needed here.

    hey pk...
    once you get one '&'...how do you not have 2 '=' present in the url?
  20. #11
  21. Head Honcho
    SEO Chat Adventurer (500 - 999 posts)

    Join Date
    Jul 2003
    Location
    Chicago, IL
    Posts
    556
    Rep Power
    16
    I agree with relaxzoolander, the only SEO to use TI quotes as a sig...
  22. #12
  23. Contributing User
    SEO Chat Genius (4000 - 4499 posts)

    Join Date
    May 2003
    Location
    Brighton
    Posts
    4,103
    Rep Power
    23
    Your right relaxzoolander. DOH!! Speed reply.
    -PK
  24. #13
  25. No Profile Picture
    K_D
    Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Jan 2004
    Posts
    119
    Rep Power
    16
    Also consider changing the title of the product pages to include the product name. This way searchers will know they've found a direct selling page when you appear in the SERPs if nothing else.

    K_D
  26. #14
  27. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Jan 2004
    Posts
    49
    Rep Power
    16
    you should use mod_rewrites within your server .htaccess files to map URL's to your dynamic files.

    example: /show/product/244
    maps to: /showproduct.php?product_id=244

    just look for .htaccess rewrite rules on google.

    also, if you stay with question mark urls, keep it under 3 variables.

Similar Threads

  1. Google indexing past the '?' in php
    By Homer in forum Google Optimization
    Replies: 16
    Last Post: Jul 20th, 2005, 10:52 PM
  2. Google indexing php pages?
    By confiden in forum Google Optimization
    Replies: 16
    Last Post: Nov 12th, 2004, 02:44 PM
  3. PHP and Google
    By leedslad73 in forum Google Optimization
    Replies: 3
    Last Post: Jan 17th, 2004, 11:02 AM
  4. Google API Problem - PHP
    By JRM in forum Web Design, Coding and Programming
    Replies: 4
    Last Post: Sep 30th, 2003, 10:22 AM
  5. GoogleBot FAQ
    By darkroom in forum Google Optimization
    Replies: 6
    Last Post: Mar 7th, 2003, 02:39 AM

IMN logo majestic logo threadwatch logo seochat tools logo