Stats

We started collecting anonymous performance data. I love stats and data. It takes the guessing out of everything. So these allow us to see things like startup times.

The stats are pretty noisy.. but they give us some good information. Here you can see the 0th, 25th, 50th, 75th and 99th percentiles. I'm flying blind on this shit but this seemed to be a better representation than the average.

From this we can see that some poor bastard is waiting 2 minutes for the game to startup, while some other lucky bastard is waiting 4 seconds. That's something we should try to address.

Then we have stats for compiling. How long is it taking people to compile at runtime?

Again, not too bad. Averaging around a second o two, but worst case can be over 30 seconds. How many files are people compiling per build, maybe that's why it's taking so long..

I suspect some people are obviously emptying libraries from github into their repository, which is accounting for 1000+ .cs files.  What's the split, what's the slow part of the compile?

The actual compile (emit) is the slowest by far, but our generators aren't far behind. Maybe we can optimize them. Actually reading the .cs files from disk is taking a decent chunk too. We could take a look at that, although I suspect 500ms is probably a good time for reading 1000+ .cs files from a presumably slow disk.

While these metrics are almost useless in isolation, they become important when you consider that we're crawling towards a release and should be making everything better. We should be able to review these charts every month and be able to see that we're improving, or at a minimum standing still.