This is a comment that I originally posted (also under my own real name) at another forum, www.up2speed.com, on February 23, 2003:
You might be asking the question, how good is the information about web traffic that Alexa provides. Alexa data is based on a statistical sample consisting of website users who have downloaded the Alexa Toolbar.
Leaving aside the question of whether these users are a random sample of all website users, which they almost certainly are not, another question is whether Alexa's sample size is really large enough to provide an accurate estimate of web traffic to a given site.
Alexa, owned by Amazon, does not provide information about their total population of toolbar users, but an analysis of Alexa data reveals this information by other means. Alexa claims to have had over 10 million downloads since the Toolbar's inception, but this provides an exaggerated picture of the toolbar's user population.
One of the two breakdowns of traffic is the Alexa Reach Rank, which purports to show the number of users per 1 million that access the site in a given period. Alexa provides reach rank data for the day, previous week, and previous three months.
It is the daily Reach Rank that provides an easy method of determining the number of users. In 1908, the physicist Robert Millikan performed a classic experiment to measure the charge of the electron, one of the fundamental physical constants in the universe. He did so by determining at what charges individual oil drops would "float" in an electrical field. Those charges at which particles were observed to float were all multiples of the charge of a single electron. For this insight, Millikan received the Nobel Prize in Physics.
Millikan's methodology is easy to use to tell the size of the Alexa sample on a given day. Just look at the daily reach rank for a large number of sites with ranks below 20,000. Those sites that were not visited in a given day have no Reach Rank for that day. A large number of sites will be visited only once, or twice. Find the reach rank of sites that are visited just once. Then divide one million by that number. The result is the Alexa sample size for a given day.
On a recent day I checked, and found that the Alexa sample size was 180,000. This makes it likely that, if you are a heavy Alexa Toolbar user, and you have a site of your own, your own browsing on your own site and others you pay attention to will bias Alexa's results significantly.
I could provide some personal anecdotal information about this, but I don't want to identify my site here and cause my results to become biased. Instead, I am linking my name to a good page on the Millikan Oil Drop Experiment. My advice is, if you need accurate ranking results, keep the Alexa browser off your own computer. Comments are welcome.