How accurate is Alexa

This is a comment that I originally posted (also under my own real name) at another forum, www.up2speed.com, on February 23, 2003:

BEGIN

You might be asking the question, how good is the information about web traffic that Alexa provides. Alexa data is based on a statistical sample consisting of website users who have downloaded the Alexa Toolbar.

Leaving aside the question of whether these users are a random sample of all website users, which they almost certainly are not, another question is whether Alexa's sample size is really large enough to provide an accurate estimate of web traffic to a given site.

Alexa, owned by Amazon, does not provide information about their total population of toolbar users, but an analysis of Alexa data reveals this information by other means. Alexa claims to have had over 10 million downloads since the Toolbar's inception, but this provides an exaggerated picture of the toolbar's user population.

One of the two breakdowns of traffic is the Alexa Reach Rank, which purports to show the number of users per 1 million that access the site in a given period. Alexa provides reach rank data for the day, previous week, and previous three months.

It is the daily Reach Rank that provides an easy method of determining the number of users. In 1908, the physicist Robert Millikan performed a classic experiment to measure the charge of the electron, one of the fundamental physical constants in the universe. He did so by determining at what charges individual oil drops would "float" in an electrical field. Those charges at which particles were observed to float were all multiples of the charge of a single electron. For this insight, Millikan received the Nobel Prize in Physics.

Millikan's methodology is easy to use to tell the size of the Alexa sample on a given day. Just look at the daily reach rank for a large number of sites with ranks below 20,000. Those sites that were not visited in a given day have no Reach Rank for that day. A large number of sites will be visited only once, or twice. Find the reach rank of sites that are visited just once. Then divide one million by that number. The result is the Alexa sample size for a given day.

On a recent day I checked, and found that the Alexa sample size was 180,000. This makes it likely that, if you are a heavy Alexa Toolbar user, and you have a site of your own, your own browsing on your own site and others you pay attention to will bias Alexa's results significantly.

I could provide some personal anecdotal information about this, but I don't want to identify my site here and cause my results to become biased. Instead, I am linking my name to a good page on the Millikan Oil Drop Experiment. My advice is, if you need accurate ranking results, keep the Alexa browser off your own computer. Comments are welcome.

END

A sample size of about 180K on a given day would be a nice sample if it were a random sample, but Alexa users are an incredibly biased selection of Internet users. Many Alexa users are webmasters and marketing junkies.

They install it on their own systems, and their own heavy use of their own pages causes their Alexa page rank to rise. Then they like Alexa, of course, because it is telling them what they want to hear.

I have been monitoring my own site, which sells art materials, for nearly two years. Initially, before I ever installed the Alexa toolbar, its rank was about 20000. I installed the toolbar on both my home and office PCs about March 2002. Over the next six months, my ranking climbed from about 13000 to 7000. I could even see the periods I was reviewing my site heavily showing up as peaks and valleys.

After I removed the toolbar from both my systems, in August 2002, it fell steadily for several months, ultimately stabilizing and returning to seasonal patterns that are normal for the art supply industry.

I do use Alexa to tell me how the next guy is doing, and the one after that. But there are many other sources of data about competitors: sample orders, cash register tapes, surveys. For those who want to pay for the data, Nielson runs a web traffic report, although I doubt it is much better than Alexa's.

There is one competitor in particular that I watch, over my shoulder, for any signs that he is gaining on us. He is a member of Bizrate, so I can count the number of customers that take his survery each month, and compare it to the number that take our survey.

Now that hundreds of webmasters and SEO optimizers are watching Alexa, it probably has even less value.

Very Interesting piece.

Truth is, although Alexa is NOT accurate, and CAN be manipulated (especially in the 80,000-200,000 range), It is still something we can rely on when initially looking at a site, as well as looking for progress - to see how we climb up (down in this case) the ladder.

True, a site with 100K is pretty similar to 80k and to 120k, but in my experience if you see a site with 250k and worse - it is defenitly a sign of no traffic, or very little.

Another thing-
The scale is exponential - it's much harder to climb up the road and the slope is increasing.

We use alexa to keep track of our ratings vs. those of competing sites. As the sites most likely draw from the same pool of users, the sample pool should be similar, so, hopefully, some of the biases are reduced.

We also find in general, Alexa correlates well with our own traffic levels. Recently, one of our sites went on a nice 3 month increase in traffic due to better PR and improved serps. Alexa also reflected this growth within 4%. The absolute numbers were not right but the relative growth numbers were close.

I think alexa is a reasonable tool to use for traffic trends but woefully inaccurate for actual ranking and visitors. My own site is 6,300 odd and it is that beacuse i tis an onlin emarketing site which of course appeals to alexa users. I dont pretend my ranking is justified an din truth if this forum was on game consoles it wouldnt be as high as it is either.

I have it installed and as mentioned it is useful for comparing sites of the same theme. What is really crass is that I'm top 70 for all german websites which is pretty mind boggling and as much as like to believe it, my traffic aint that good ;-)

Talking of Alexa ranking. Congratulations SEOChat for breaking the 1,500 milestone :-)

What about the google toolbar? Does it report to alexa....

No, the google toolbar does not have any relation to Alexa.

I hate Alexa as much as the next person, but being able to use "Our Site is ranked 10,000 out of 300,000,000,000" as selling point is awesome.

Jon makes a good point. Like I have said numerous times, for some reason prospective advertisers DO care what your Alexa ranking is, even if it is not a completely accurate depiction of traffic.

I have a question regarding Alexa's rankings for our company's suite of websites. I fully understand the discussions above and we use Alexa as a "gauge" of traffic, and understand that it can be completely manipulated by simply installing more Alexa toolbars on PC's and having those users visit your site(s).

Our websites are:
www.ForeclosureNet.net
www.NJPForeclosures.com
www.BankForeclosureList.com
www.BankHouses.com
www.TREL.com (yes, we know this one is a dinosaur and desperately needs updating!)

We have noticed over the course of the past 2-3 months that our Alexa ranking is plummeting on ALL of our sites. We lost one huge partner 2-3 months ago that was sending traffic to only ONE of our sites, www.ForeclosureNet.net . It would make complete sense to me if our Alexa ranking dropped on ForeclosureNet because of that significant loss of traffic, however, our Alexa ranking has been dropping on our whole suite of sites. The only major change to any of our sites was the one partner that I mention above that stopped sending traffic to ONE of our sites. What's even weirder, is that according to our WebStats that we use, our overall sites traffic does not really seem to be decreasing that much, other than a bit of seasonality decrease, but there is not much change in traffic across all of our sites.

At the exact same time all of our sites started dropping in Alexa ranking, all of our competitors Alexa rankings started rising as quickly as we were falling. Therefore, the ranking cannot be attributed to less interest in our "product" because obviously plenty of people are still visiting our competitors website. Our competitor, www.ForeclosureFreeSearch.com also uses very deceitful spamming techniques, where they define "Blockquote" in their CSS sheets to = a value of "do not display", so everywhere on their page where they have the word "Blockquote", they stuff PAGES of text that don't show up on the actual website because they are hidden. Do a view source on their site to see what I am talking about. With these practices, they will obvioulsy rank higher than us in the SE's (although I cannot believe Google has not penalized them for this yet). I am not sure if this has anything to do with the problem at hand though.

Even about 5 of our other competitors Alexa rankings are increasing while all of our websites A.Rankings decrease. What can account for this?

Thanks in advance for your responses.

1. It makes sence that if your competitors are rising you're falling.... your audience is going to their sites!
2. what's up with your actual traffic - has it also fell down or just the alexa rank?
3. Regards spammers - snitch without thinkinng twice. these kinds of expoits are killing the industry as it might be too hard for G to tell what is good CSS and what is bad and ban everything!

Thanks for your response Ophric. Our acutal traffic is the same and even slightly increasing. That is what is so weird about our Alexa ranking decreasing...

Regarding the CSS Spamming techniques....We've tried to report them on multiple occasions and nothing ever happens.

I only have an alexa rating of 600,000 but i get well over 1,000 unique visitors per day... I wouldnt call that very little traffic...

Interesting...
Where is your traffic coming from? (like countries, user demographics)? it might help understand stuff...

And...
A friend offered a theory suggesting that since alexa is Amazon owned company, they might be sharing informationv - thus using amazon users, tracking their behavior to determine the high ranked sites... any thoughts?

Hi,

The site in question is URL

The demographics for the last week are made up as follows:

North America - (41.9% of all sessions)
Europe - (17.6% of all sessions)
Australia and Oceania - (4.0% of all sessions)
Asia - (3.7% of all sessions)
South America - (0.9% of all sessions)
Africa - (0.4% of all sessions)

I think this looks like a pretty average spread of visitors.

Interesting...
I guess your visitors are mostly home users? not work, hi tech and programming oriented and webmasters and far east where alexa toolbar is most used...

I does kill my amazon theory though...