Search Engine Spiders
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
User Name:
Password:
Remember me
Go Back   SEO Chat ForumsSearch Engine StrategiesSearch Engine Spiders

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread SEO Chat Forums Sponsor:
  #1  
Old October 15th, 2009, 10:01 AM
Wyatte Wyatte is offline
Registered User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Oct 2009
Posts: 4 Wyatte User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 7 m 25 sec
Reputation Power: 0
Php files and robots.txt

I have a database driven site I'm working on for a client. They did not have a robots txt file so I'm adding one.

Most all urls are seo friendly, however the regular php files are still on the server and could be picked up from the spiders.

So, should I exclude product.php, contact.php, etc... to the robots.txt?

Thanks

Reply With Quote
  #2  
Old October 15th, 2009, 10:08 AM
jsteele823's Avatar
jsteele823 jsteele823 is offline
Search Satisfaction
Click here for more information. Click here for more information
Click here for more information
 
Join Date: May 2007
Location: Georgia, USA
Posts: 2,232 jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level) 
Time spent in forums: 1 Month 1 Week 2 Days 18 h 6 m 59 sec
Reputation Power: 243
Send a message via Yahoo to jsteele823 Send a message via Skype to jsteele823
If you have friendly URLs, then most likely those php pages are 301 redirected to the friendlies, right? If that's the case, then no, you don't need to exclude them.
__________________

"To repeat what others have said, requires education; to challenge it, requires brains." - Mary Pettibone Poole, A Glass Eye at a Keyhole, 1938
“The wise understand by themselves; fools follow the reports of others” - Tibetan proverb

SEOChat FAQ's - A good place to start learning
Please read this before posting.

Reply With Quote
  #3  
Old October 15th, 2009, 11:33 AM
Wyatte Wyatte is offline
Registered User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Oct 2009
Posts: 4 Wyatte User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 7 m 25 sec
Reputation Power: 0
Thanks for the reply, although, they are rewritten, I just noticed they aren't redirected,

take a look at this page/url
Code:
http://www.dieselgeneratorsmiami.com/productos_FC.php?hz=60&ph=1&kwi=7&kwf=150


Look toward the bottom of the page where the « previous 1 | 2 | 3 next »

hover over and you'll see those are rewritten.

Any suggestions on how to proceed? Thanks again.

Reply With Quote
  #4  
Old October 15th, 2009, 11:38 AM
googler's Avatar
googler googler is offline
Cool Dude
SEO Chat Regular (2000 - 2499 posts)
 
Join Date: Aug 2003
Location: Vancouver, Washington, U.S.A.
Posts: 2,198 googler User rank is First Lieutenant (10000 - 20000 Reputation Level)googler User rank is First Lieutenant (10000 - 20000 Reputation Level)googler User rank is First Lieutenant (10000 - 20000 Reputation Level)googler User rank is First Lieutenant (10000 - 20000 Reputation Level)googler User rank is First Lieutenant (10000 - 20000 Reputation Level)googler User rank is First Lieutenant (10000 - 20000 Reputation Level)googler User rank is First Lieutenant (10000 - 20000 Reputation Level)googler User rank is First Lieutenant (10000 - 20000 Reputation Level) 
Time spent in forums: 2 Weeks 3 Days 23 h 9 m 21 sec
Reputation Power: 177
Send a message via AIM to googler Send a message via XFire to googler
the products.php and contact.php currently returns a 404. This is fine... If it has been rewritten from the beginning I see no problem with this method. No blocking/redirecting necessary
__________________
Work Smarter, Not Harder
The Definitive Guide to Page Rank
Link Building 101
PPC 101

Reply With Quote
  #5  
Old October 15th, 2009, 11:46 AM
Wyatte Wyatte is offline
Registered User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Oct 2009
Posts: 4 Wyatte User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 7 m 25 sec
Reputation Power: 0
Quote:
Originally Posted by googler
the products.php and contact.php currently returns a 404. This is fine... If it has been rewritten from the beginning I see no problem with this method. No blocking/redirecting necessary


Those were just an example, they don't actually exist. Notice the url I posted on the third post.

Reply With Quote
Reply

Viewing: SEO Chat ForumsSearch Engine StrategiesSearch Engine Spiders > Php files and robots.txt


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump




 Free IT White Papers!
 
Create the Optimal Architecture for your Critical Applications
Warburton's the largest independently owned bakery in the UK faced a number of difficult challenges in providing the most robust yet efficient IT infrastructure for their organization's success. IBM's services combined with their xSeries servers created the perfect platform for their SAP environment with sufficient flexibility, and did so in very time effective fashion.

Request Your Free Technology Downloads!
 
Five Best Practices for Deploying a Successful Service-Oriented Architecture
This white paper describes the benefits you can expect with SOA, and how IBM can help take your business there.

Request Your Free Technology Downloads!
 
Gartner Magic Quadrant for Application Delivery Controllers
Gartner summarizes its view on Application Delivery Controllers, evaluates strengths and weaknesses of solutions, and provides Magic Quadrant reporting for a quick comparison across all vendors. Learn from Gartner how you can benefit from an all-in-one device like Citrix NetScaler that delivers the highest levels of availability, performance and security.

Request Your Free Technology Downloads!
 
Knowledge is Power
What you don't know can hurt you, and is likely costing you money and increasing your security risks during an era of scarce resources. This white paper proposes six key strategies that enterprise security managers can use to improve their network defense posture.

Request Your Free Technology Downloads!
 
Rationalizing the Multi-Tool Environment
The rationalized multi-tool approach is flexible, scalable and cost effective. It provides the necessary input to the IT service management business processes. It preserves prior investments in monitoring tools, empowers technologists to select the best tools with which to do their jobs, and enhances effective response to incidents.

Request Your Free Technology Downloads!
 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 




© 2003-2010 by Developer Shed. All rights reserved. DS Cluster 7 Hosted by Hostway
For more Enterprise Application Development news, visit eWeek