#1
  1. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Sep 2004
    Posts
    35
    Rep Power
    15

    .htaccess, dynamic urls, and google


    I just created a new website - cgi based and database driven. If fully indexed by google it should have about 100,000 webpages or so. I masked my urls with .htaccess in order to make my site 'appear' static. The urls went from this:

    http://www.mysite.com/a.cgi?a=widget

    to this:

    http://www.mysite.com/shop/widget

    So as you can see, the second url looks and seems a lot better. I was just wondering if anybody knows why google will index the latter so much easier than the first url. Google cannot tell that they both come from a dynamic page generator? Will my site index a lot faster because of .htaccess? This is my first website I've created using this feature.....and my website just went live this morning.
  2. #2
  3. No Profile Picture
    User Contributor
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Aug 2004
    Posts
    317
    Rep Power
    15
    When you use something that is server side like PHP or .htaccess, everything takes place then is sent to the browser...try to view this pages source...you won't see any php code, even though it is there.

    How fast your total pages get indexed will be based on your internal link structure and your inbound links.
    Google doesn't know your site is dynamic, G just thinks you have directories and sub directories.
  4. #3
  5. I <3 Google
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Jun 2004
    Location
    Arizona
    Posts
    216
    Rep Power
    15
    askjoe im using php and my urls are almost identical!!

    myurls.com/product.php?id=$id


    How do change them to....
    myurls.com/$id.html ?
  6. #4
  7. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Oct 2004
    Posts
    4
    Rep Power
    0

    Smile Dyanamic page Indexing


    Originally Posted by askjoe
    I just created a new website - cgi based and database driven. If fully indexed by google it should have about 100,000 webpages or so. I masked my urls with .htaccess in order to make my site 'appear' static. The urls went from this:

    http://www.mysite.com/a.cgi?a=widget

    to this:

    http://www.mysite.com/shop/widget

    So as you can see, the second url looks and seems a lot better. I was just wondering if anybody knows why google will index the latter so much easier than the first url. Google cannot tell that they both come from a dynamic page generator? Will my site index a lot faster because of .htaccess? This is my first website I've created using this feature.....and my website just went live this morning.
    Hi askjoe]1

    Actually whenever indexing by Google is done in case of dynamic pages, firstly the request for a particular page is given to a server. If any Scripting has been used , it is submitted in the form of a Query. Then Interpretor on the server side solves the query and then produces a HTML page. So, the indexing is difficult or more cumbersome in cases of URL with Query strings. No doubt that your site with .htaccess will be better indexed by google.

    Gaurav
  8. #5
  9. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Sep 2004
    Location
    Washington D.C.
    Posts
    52
    Rep Power
    15
    Originally Posted by AfterHim.com
    When you use something that is server side like PHP or .htaccess, everything takes place then is sent to the browser...try to view this pages source...you won't see any php code, even though it is there.

    How fast your total pages get indexed will be based on your internal link structure and your inbound links.
    Google doesn't know your site is dynamic, G just thinks you have directories and sub directories.
    Thats not completely correct. Google does reconize that finding a ? in a url mostly likely means the site is dynamic and will slow down the crawl in order to not "bombard" or crash your site. That being said, I couldn't tell you wether that turns a 30 minute crawl into a 6 hour or if google comes back on different days to finish it. I suspect it's a combination of both because my vbulletin forum with some 15k posts gets hit with a deep crawl for literally days at a time with 8 - 12 hour pauses while my higher ranked sites using static url's ala htaccess get deep crawled much faster.

    Theere is a reference to dynamic pages at http://www.google.com/webmasters/2.html and there was a interview with a google engineer, Daniel Dulitz, in December of 2003 where he said:
    We deal very well with dynamic delivery per se. So
    the mere presence of a question mark in the URL doesn't
    throw us for a loop at all. We will crawl those pages more
    slowly than we do with pages without question marks in the
    URL. Purely because they've identified themselves as being
    dynamic and we certainly don't want to bring anyone's site
    down because we're hitting the database too hard.
  10. #6
  11. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Sep 2004
    Posts
    35
    Rep Power
    15
    I thought google considered dynamic pages inferior to static pages because of the ease in which content can change. Take news sites for example......google comes, spiders a page with a particular headline, by the time the page has made it into google a new headline has appeared thus making google's search results innaccurate.

    While my site doesn't deal with news and won't be changing dramatically everyday, this was the primary reason for me to study up on and apply .htaccess - to make my pages appear static and thus achieve a 'static' ranking. I just wanted to know if others have experienced increased SEO because of this feature. It's one thing to read articles, it's another to hear it first hand.
  12. #7
  13. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Sep 2004
    Location
    Washington D.C.
    Posts
    52
    Rep Power
    15
    No, google doesn't penalize dynamic site or favor them over static ones. The web has changed alot in the last 2 years and sites built using php, asp or cgi are far from a rarity anymore. The popularity and high ranking of many blogs which use query strings is a great example of this.

    The only place it makes a difference is wether your site is well coded. A page like index.php?q=widget will rank better than index.php?q=5 for widget. Also avoiding things like session id's in a url will prevent your site from getting many pages left out and allow pagerank to flow smoothly.

    In your example above if you changed a.cgi to shop.cgi you would get the same keyword benefit as you are with the htaccess but with a slower crawl time. So it's a matter of taste and you may very well do better on the other engines with the htaccess.
    Last edited by WColby; Oct 13th, 2004 at 03:46 PM.

Similar Threads

  1. google & dynamic url
    By happyallday in forum Google Optimization
    Replies: 10
    Last Post: Jun 30th, 2004, 08:12 AM
  2. Forums that do not utilize dynamic URLs
    By PanCorp in forum Google Optimization
    Replies: 12
    Last Post: Jun 9th, 2004, 05:27 PM
  3. how good is Google at spidering dynamic sites?
    By puterbug in forum Google Optimization
    Replies: 8
    Last Post: Jun 23rd, 2003, 09:16 AM
  4. GoogleBot FAQ
    By darkroom in forum Google Optimization
    Replies: 6
    Last Post: Mar 7th, 2003, 02:39 AM

IMN logo majestic logo threadwatch logo seochat tools logo