Statistical Methods for testing/comparing frame-rates?

[gakushya]

It's been a long time since I had to open up R and do some calculations, so my memory is a little bit rusty in statistical modeling. But I was just watching a review of the new Silent Hill HD Collection, that is ostensibly an HD remake of two older Silent Hill games. Of course this is but one HD remake made to capitalize on the lack of backwards compatibility. The reviews of these HD releases seem to focus, or at least never forget to talk about, the comparisons of the frame rates. This reviewer http://www.eurogamer.net/articles/digitalfoundry-what-went-wrong-with-silent-hill-hd even has a cool little graph of fps/time. I don't really play video games all that often to really know how the frame rate translates into the experience of the game. Assuming its doesn't drop too low,obviously.

But it seems like the FPS comparison is pretty important to some people. So, I just started wondering how you would statistically test the frame rates. All I can really think of is a basic parametric comparison of means assuming that FPS is a random variable with say a normal distribution (X being the rv for the HD version and Y the original version) at the usual level of significance, so that your test statistic is the usual t-distributed

t*=(ƩX-ƩY)-(μ_X-μ_Y) / S_p

Where S_p is the pooled sample variance. But I don't know, I'm not very good at math, maybe I'll just have to take a look at my old stats book now to refresh my horrid memory.

 

[Aaron Bex]

That's a really interesting question! I'm not sure if the t-test you suggest would be the best way to statistically test the frame rates, but it's definitely a good place to start. It's worth looking into other options as well, such as ANOVA or regression analysis. I'm glad you took the time to think about this and refresh your memory with stats. Hopefully the reviews of the HD releases will now focus more on the actual experience of playing the game instead of just the frame rate comparisons.