A lot of questions still exist about the Meta robots tag and robots.txt despite the popularity of these terms with any search engine optimization professional. In this two-part series we will attempt to answer these questions scientifically by doing an experiment and collecting data.

Read the full article here: How Search Engines Handle the Meta Robots Tag and Robots.txt

For more discussion go here: Blog Article Discussion