Google Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
User Name:
Password:
Remember me
Go Back   SEO Chat ForumsGoogleGoogle Optimization

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread SEO Chat Forums Sponsor:
  #1  
Old December 6th, 2008, 10:27 AM
poweri poweri is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Aug 2006
Posts: 39 poweri User rank is Corporal (100 - 500 Reputation Level)poweri User rank is Corporal (100 - 500 Reputation Level)poweri User rank is Corporal (100 - 500 Reputation Level)poweri User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 11 h 28 m 11 sec
Reputation Power: 5
Scrape Google pages using curl = disadvantages?

I want to use curl to scrape pages from Google search engine. Of course people will see that the pages are from Google, but I want to have the pages embedded in my website.

Can they blacklist the server of your host for fetching the data? I suppose they (Google server,...) can see the fetching and because I have read something about limited use for websites and software developers (1000 hits a day?), I have this kind of questions. I don't want trouble...

Reply With Quote
  #2  
Old December 6th, 2008, 10:34 AM
JagNet's Avatar
JagNet JagNet is offline
Smoke me a kipper...
SEO Chat Regular (2000 - 2499 posts)
 
Join Date: Aug 2007
Posts: 2,487 JagNet User rank is Second Lieutenant (5000 - 10000 Reputation Level)JagNet User rank is Second Lieutenant (5000 - 10000 Reputation Level)JagNet User rank is Second Lieutenant (5000 - 10000 Reputation Level)JagNet User rank is Second Lieutenant (5000 - 10000 Reputation Level)JagNet User rank is Second Lieutenant (5000 - 10000 Reputation Level)JagNet User rank is Second Lieutenant (5000 - 10000 Reputation Level)JagNet User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 1 Month 2 Weeks 4 Days 3 h 37 m 22 sec
Reputation Power: 68
Yes, they can deny your IP access for this, after all it's against Google TOS and you're basically stealing their data.

If you want to make use of Google search results in your pages there are legitimate methods, such as their search APIs.
__________________
... I'll be back for breakfast
New to SEO? SEOChat SEO FAQs
Forum Rules and Posting Guidelines
URL canonicalization code solutions
Vigorously pursuing the floccinaucinihilipilification of cheap SEO tricks

Reply With Quote
  #3  
Old December 6th, 2008, 10:47 AM
poweri poweri is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Aug 2006
Posts: 39 poweri User rank is Corporal (100 - 500 Reputation Level)poweri User rank is Corporal (100 - 500 Reputation Level)poweri User rank is Corporal (100 - 500 Reputation Level)poweri User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 11 h 28 m 11 sec
Reputation Power: 5
Quote:
Originally Posted by JagNet
Yes, they can deny your IP access for this, after all it's against Google TOS and you're basically stealing their data.

If you want to make use of Google search results in your pages there are legitimate methods, such as their search APIs.

I see you have to use javascript? And that sounds like Google want to control the use and of course with limits etc.

Is there another solution for bypassing a Google check?

Reply With Quote
Reply

Viewing: SEO Chat ForumsGoogleGoogle Optimization > Scrape Google pages using curl = disadvantages?


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump




 Free IT White Papers!
 
Create the Optimal Architecture for your Critical Applications
Warburton's the largest independently owned bakery in the UK faced a number of difficult challenges in providing the most robust yet efficient IT infrastructure for their organization's success. IBM's services combined with their xSeries servers created the perfect platform for their SAP environment with sufficient flexibility, and did so in very time effective fashion.

Request Your Free Technology Downloads!
 
Five Best Practices for Deploying a Successful Service-Oriented Architecture
This white paper describes the benefits you can expect with SOA, and how IBM can help take your business there.

Request Your Free Technology Downloads!
 
Gartner Magic Quadrant for Application Delivery Controllers
Gartner summarizes its view on Application Delivery Controllers, evaluates strengths and weaknesses of solutions, and provides Magic Quadrant reporting for a quick comparison across all vendors. Learn from Gartner how you can benefit from an all-in-one device like Citrix NetScaler that delivers the highest levels of availability, performance and security.

Request Your Free Technology Downloads!
 
Knowledge is Power
What you don't know can hurt you, and is likely costing you money and increasing your security risks during an era of scarce resources. This white paper proposes six key strategies that enterprise security managers can use to improve their network defense posture.

Request Your Free Technology Downloads!
 
Rationalizing the Multi-Tool Environment
The rationalized multi-tool approach is flexible, scalable and cost effective. It provides the necessary input to the IT service management business processes. It preserves prior investments in monitoring tools, empowers technologists to select the best tools with which to do their jobs, and enhances effective response to incidents.

Request Your Free Technology Downloads!
 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 




© 2003-2010 by Developer Shed. All rights reserved. DS Cluster 5 Hosted by Hostway
For more Enterprise Application Development news, visit eWeek