Google Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
User Name:
Password:
Remember me
Go Back   SEO Chat Forums > Google > Google Optimization
Reload this Page

Help in Robots.txt

Discuss Help in Robots.txt in the Google Optimization forum on SEO Chat.
Help in Robots.txt Google Optimization forum discussing techniques to increase rank in Google. Find help for beating spam pages, getting out of the sandbox, and beating the Google algorithm. Also read about the best SEO tools to help with Google search.

Record onscreen activity, your voice & webcam video. Turn everyday presentations into HD-quality videos.

For perfectly clear screencasts every time.

We cover the world of search engine optimization like no one else, constantly updating you with the best information and news from around the world.

Take your business to the next level, with "The Edge of Success: 9 Building Blocks to Double Your Sales." Inside, you will discover how to easily capture more leads, get repeat sales, and significantly grow your business.


Try Camtasia Studio today!
Sign Up today!
Download this FREE ebook now!

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
« Previous Thread | Next Thread »  
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread SEO Chat Forums Sponsor:
  #1  
Old November 17th, 2009, 06:11 AM
madeline madeline is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Jul 2009
Posts: 44 madeline User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 13 h 47 m 3 sec
Reputation Power: 1
Help in Robots.txt
Hi,

I have one important url which is not indexed in Google. To make it indexed faster can i add it in Robots.txt under allow.
Reply With Quote
  #2  
Old November 17th, 2009, 06:15 AM
Jim1 Jim1 is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Jun 2009
Posts: 171 Jim1 User rank is Sergeant (500 - 2000 Reputation Level)Jim1 User rank is Sergeant (500 - 2000 Reputation Level)Jim1 User rank is Sergeant (500 - 2000 Reputation Level)Jim1 User rank is Sergeant (500 - 2000 Reputation Level)Jim1 User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 5 Days 7 h 28 m 22 sec
Reputation Power: 15
Quote:
Originally Posted by madeline
Hi,

I have one important url which is not indexed in Google. To make it indexed faster can i add it in Robots.txt under allow.



Hi madeline,

No I THINK YOUR BETTER OF DOING THE BELOW
.

Get some trusted backlinks to the page.

Have you submited a xml site through google webmaster tools?????????
Last edited by Jim1 : November 17th, 2009 at 06:17 AM.
Reply With Quote
  #3  
Old November 17th, 2009, 06:15 AM
KernelPanic KernelPanic is offline
Contributing User
SEO Chat Beginner (1000 - 1499 posts)
  
Join Date: Aug 2008
Posts: 1,070 KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level) 
Time spent in forums: 1 Week 4 Days 12 h 46 m 3 sec
Reputation Power: 154
Quote:
Originally Posted by madeline
Hi,

I have one important url which is not indexed in Google. To make it indexed faster can i add it in Robots.txt under allow.


You can do that Also I would submit a sitemap.xml through Google Webmaster Tools and gain a couple of quality links.
Reply With Quote
  #4  
Old November 17th, 2009, 07:38 AM
jsteele823's Avatar
jsteele823 jsteele823 is offline
Search Satisfaction
Click here for more information. Click here for more information
Click here for more information
 
Join Date: May 2007
Location: Georgia, USA
Posts: 2,232 jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level)jsteele823 User rank is Captain (20000 - 30000 Reputation Level) 
Time spent in forums: 1 Month 1 Week 2 Days 18 h 11 m 19 sec
Reputation Power: 244
Send a message via Yahoo to jsteele823 Send a message via Skype to jsteele823
Just to clarify - the xml sitemaps and the backlinks mentioned above will do the trick.

Adding an allow directive will not change a thing. It is pointless. Everything is allowed by default.
__________________

"To repeat what others have said, requires education; to challenge it, requires brains." - Mary Pettibone Poole, A Glass Eye at a Keyhole, 1938
“The wise understand by themselves; fools follow the reports of others” - Tibetan proverb

SEOChat FAQ's - A good place to start learning
Please read this before posting.
Reply With Quote
  #5  
Old November 17th, 2009, 07:48 AM
madeline madeline is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Jul 2009
Posts: 44 madeline User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 13 h 47 m 3 sec
Reputation Power: 1
since i am running a shopping site, products are found after many clicks from the Homepage.

So I added the link in the Homepage and submitted sitemap, still not crawled.
Reply With Quote
  #6  
Old November 17th, 2009, 08:31 AM
KernelPanic KernelPanic is offline
Contributing User
SEO Chat Beginner (1000 - 1499 posts)
  
Join Date: Aug 2008
Posts: 1,070 KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level) 
Time spent in forums: 1 Week 4 Days 12 h 46 m 3 sec
Reputation Power: 154
Quote:
Originally Posted by madeline
since i am running a shopping site, products are found after many clicks from the Homepage.

So I added the link in the Homepage and submitted sitemap, still not crawled.


wow, that was almost an hour ago. I wonder what could be wrong....


Just Kidding!

Give it time, you'll see. In the meantime, concentrate on building Q U A L I T Y inbound links to that page.
Reply With Quote
  #7  
Old November 17th, 2009, 09:47 PM
Traffic-Bug Traffic-Bug is offline
Permanently Banned
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Mar 2009
Location: Oregon
Posts: 361 Traffic-Bug Negative: is most likely a SPAMMER and a traitor to the cause. 
Time spent in forums: 2 Days 8 h 59 m 39 sec
Warnings Level: 10
Number of bans: 3
Reputation Power: 0
Facebook
Without taking any chances, I would add one Allow entry in the robots.txt for the directory that page is in, and submit a sitemap including that page and make sure that page is well linked-to internally.
Reply With Quote
  #8  
Old November 17th, 2009, 10:01 PM
studyhere6 studyhere6 is offline
Registered User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Nov 2009
Posts: 4 studyhere6 User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 5 m 38 sec
Reputation Power: 0
Quote:
Originally Posted by madeline
Hi,

I have one important url which is not indexed in Google. To make it indexed faster can i add it in Robots.txt under allow.


Hi, if i have the newpage, i will add in my sitemap.xml. and you need get more backlinks to this page, maybe google will craw your url and indexed it
Reply With Quote
  #9  
Old November 19th, 2009, 04:37 AM
Web Marketer Web Marketer is offline
Registered User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Nov 2009
Posts: 39 Web Marketer Negative: is most likely a SPAMMER and a traitor to the cause. 
Time spent in forums: 6 h 4 m 22 sec
Reputation Power: 0
reply
i would suggest that give some time till then concentrate on your inbound linking strategy.......
Reply With Quote
  #10  
Old November 19th, 2009, 04:53 AM
marky marky is offline
Registered User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Aug 2008
Posts: 23 marky Negative: is most likely a SPAMMER and a traitor to the cause. 
Time spent in forums: 8 h 2 m 49 sec
Reputation Power: 0
hi

you don't need to update in robots.txt . you need to create more and more quality link which support your site in indexing and also in ranking.
Reply With Quote
  #11  
Old November 19th, 2009, 06:02 AM
madeline madeline is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Jul 2009
Posts: 44 madeline User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 13 h 47 m 3 sec
Reputation Power: 1
I found that GoogleBot has not visited my site for the past one week and thats why the url which I place in the Homepage is not indexed.

I dont't know why Google Bot has not visited. mine is a dynamic site and contents are updated regularly.

Google Bot visits my site once in 2 days and now it is not.

What may be the reason?
Reply With Quote
  #12  
Old November 19th, 2009, 06:16 AM
KernelPanic KernelPanic is offline
Contributing User
SEO Chat Beginner (1000 - 1499 posts)
  
Join Date: Aug 2008
Posts: 1,070 KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level)KernelPanic User rank is First Lieutenant (10000 - 20000 Reputation Level) 
Time spent in forums: 1 Week 4 Days 12 h 46 m 3 sec
Reputation Power: 154
Quote:
Originally Posted by madeline
I found that GoogleBot has not visited my site for the past one week and thats why the url which I place in the Homepage is not indexed.

I dont't know why Google Bot has not visited. mine is a dynamic site and contents are updated regularly.

Google Bot visits my site once in 2 days and now it is not.

What may be the reason?


Don't stress over that which you cannot control. Instead, take the above advice.
Reply With Quote
  #13  
Old November 19th, 2009, 06:33 AM
Notorious1 Notorious1 is offline
Registered User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Nov 2009
Posts: 1 Notorious1 User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 18 m 56 sec
Reputation Power: 0
Some very useful information.
Reply With Quote
  #14  
Old November 19th, 2009, 06:38 AM
alenjolly alenjolly is offline
Registered User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Nov 2009
Location: USA
Posts: 18 alenjolly User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 3 h 56 m 20 sec
Reputation Power: 0
Thumbs up
Don't be excited, just try to submit your site to DOFOLLOW social bookmarking websites...
Reply With Quote
Reply

Viewing: SEO Chat Forums > Google > Google Optimization > Help in Robots.txt

« Previous Thread | Next Thread »

Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Linear Mode Linear Mode
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump




 Free IT White Papers!
 
Create the Optimal Architecture for your Critical Applications
Warburton's the largest independently owned bakery in the UK faced a number of difficult challenges in providing the most robust yet efficient IT infrastructure for their organization's success. IBM's services combined with their xSeries servers created the perfect platform for their SAP environment with sufficient flexibility, and did so in very time effective fashion.
Request Your Free Technology Downloads!
 
Five Best Practices for Deploying a Successful Service-Oriented Architecture
This white paper describes the benefits you can expect with SOA, and how IBM can help take your business there.
Request Your Free Technology Downloads!
 
Gartner Magic Quadrant for Application Delivery Controllers
Gartner summarizes its view on Application Delivery Controllers, evaluates strengths and weaknesses of solutions, and provides Magic Quadrant reporting for a quick comparison across all vendors. Learn from Gartner how you can benefit from an all-in-one device like Citrix NetScaler that delivers the highest levels of availability, performance and security.
Request Your Free Technology Downloads!
 
Knowledge is Power
What you don't know can hurt you, and is likely costing you money and increasing your security risks during an era of scarce resources. This white paper proposes six key strategies that enterprise security managers can use to improve their network defense posture.
Request Your Free Technology Downloads!
 
Rationalizing the Multi-Tool Environment
The rationalized multi-tool approach is flexible, scalable and cost effective. It provides the necessary input to the IT service management business processes. It preserves prior investments in monitoring tools, empowers technologists to select the best tools with which to do their jobs, and enhances effective response to incidents.
Request Your Free Technology Downloads!
 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 




© 2003-2010 by Developer Shed. All rights reserved. DS Cluster 9 Hosted by Hostway. 
For more Enterprise Application Development news, visit eWeek