Marshall Sponder, over on Web Metrics Guru, looks into Comscore’s stats of Second Life’s users and finds the same problem.

The thing is these services rely on toolbars (I can’t even use any of the toolbars on the Macintosh for some reason, and how many of you even have one of these folks’ toolbars loaded? None of my friends do and I’ve been checking). Or they rely on “panels” of Web users that they survey regularly. Do you know the selection mechanisms? How do they know they are getting a representative sample? Clearly very few people who run Web companies find their stats accurate. Yet we’re supposed to believe in them?

Also, there are lots of sites who seem to have more traffic than, say, my blog, but they get less comments on every post and if we both link to someone new, the new site gets a lot more traffic from me. I have such a site in mind, but I don’t want to get into an argument with that site. Translation: the engagement levels on some blogs are quite different, but advertisers are being sold on these stats companies and on pure “uniques.”

I don’t know what the solution is, though. What stats do you think are the most important? What’s the most accurate way to measure your sites’ visitors? What will advertisers insist on seeing in the future?

Oh, and in the future people aren’t going to visit your page at all. Most of PodTech’s traffic comes from its embeddable gadget. So, are you visiting a blog that has our gadget embedded when you watch one of my videos or are you visiting PodTech? I bet most normal people will answer “a blog.” That’d mean that PodTech’s traffic will get way underrepresented in these services (which matches what we’re seeing in our server logs when we compare our real traffic with what Alexa/Compete/Comscore are telling us).