Saturday, December 9, 2006

Last 60 days - something very wrong

Something clearly wrong



There's a number that Linden Lab publishes on the Second Life homepage and login screen and in an XML data source. It's called 'Logged in Last 60 Days'. In theory this number is the number of distinct accounts that logged in to Second Life within the last 60 days. If you believe that it means what Linden Lab seems to believe it to mean, anyway.

The problem is, there's something seriously wrong with this number. Observe the chart on the right. Click on it to get a larger version for clarity.

That's what you get if you graph that last-60-days number less the number of people who signed up in the same period. There's some obviously perplexing and disturbing things about it.
First, are the two narrow spikes -- the data source hasn't always provided a number reliably, so you can ignore those. The thing that stands out is just left of center where the total population was restated around August 15/16 (up until that point it was supposed to be a weighted average of the last-60-day logins and total signups. After that point it's just total signups). And there's where everything goes south -- Well, quite literally, looking at the graph. Down into negative-numbers. More signups for a 60 day period than logins.

When we generated this graph last week, we rechecked all our data and contacted Linden Lab with our findings (and again, 2 hours before publishing). We've received no comment from them, but at that time -- according to their own published figures -- in the previous 60 days, three hundred thousand people had signed up for Second Life and not actually logged in within that 60 day period. That's just not right.

Now, maybe Linden Lab are doing something (or maybe the code that generates this number has just finally had a cow). The day after we sent our findings to Linden Lab, the active figure froze at 700,030 (falling slightly, for the first time) and remained there for a couple of days. Then it fell once more to it's current level of 690,800, where it is currently stuck at the time of this writing. The calculation of the figure is obviously not working properly. Granted, Linden Lab appear to have other things on their minds, but published figures like these are highly media-visible, and I'm not going to be the last to notice the perplexing discrepancies.

The most likely thing that's going on here is that the number has been being understated, either due to an error when the code was written, or due to sundry database cluster changes which may have caused some of the user data to move to servers that this statistical system can't reach.

It's also possible that the code was originally written to perform some rather different calculation, and that the figure does not represent what Linden Lab seem to think it does. In the absence of any word from Linden Lab, we can only speculate.

No comments: