Statistical Hypothesis Testing for SEO Ranking Factors?

Say you are able to conduct an SEO experiment on a small sample of real websites. One half of the sample make the intended change that you are investigating and on the control make no change.
After a suitable length of time see the results and use statistical analysis such as the student-t test to either add weight to your hypothesis or reject your hypothesis and accept the null hypothesis.
Anyone see any problems in this method of testing ranking factors? Has anyone ever tried?

1.the problem is isolating your experiments from the actions of competitors or other sites for selected words

2. I believe the table size for keyword or keyword combo is limited so you have to experiment on extremely non competive keywords.

3.the other problem is speculation but if it exists it can really screw up your analysis - that is the use of weighting (google may call them dampening) for scoring factors.

iaw score= W1 times text on page score + W2 times text in links to apge score + or * W3 time page rank score.

if google tweaks the weights even just a bit it will screw up your analysis

head nodding here...YES!!! this is exactly how you test SEO tactics...many many of us here have discussed this and practiced just this kind of process to prove/disporve our theoretical assumptions!

ie the scientific process works, eh?

:-)

Jim