Opinion - The value of QA
As a solo indie game dev, I can't stress enough how many times I've played my game during development. If I was to try to put a number to this, I't be in the thousands. Mostly because every time something has been implemented I play test to ensure it works correctly. I try every scenario I can think of when I'm play testing too. I'm fairly comfortable with knowing where my bugs are and knowing that I'll get around to fixing them later. Major bugs I find, I fix right away but the little things I come back to later in a bug fix pass.
Feeling this way, I sent out a QA build for some game design students from AIE Online (My former collage) to run through as part of their QA class. In sending the build out i thought to myself, I know where every bug is so the QA feedback will be more valuable as performance testing rather than actual bug reports. Boy was I wrong.
I mentioned before that I try to think of every possible scenario when I play test but it's become very apparent that players do really weird things. Things I can't emulate because I am incapable of thinking the same way as every other person that plays my game. If we were a team of 10+ I'm sure much of this would have been discovered but being a lonesome dev, it can be very difficult to find the obscure.
Take this bug that was reported to me. If you open the sliding doors, then close them, then open them again, the animation doesn't play correctly. I would kick myself for not testing that. I remember implementing these particular doors and would have tested it. The difference is, I tested "does it close and open"? Yes, then it works. I didn't click on them that third time. This is actually a significant bug that I may have found before shipping but what if I didn't. Whats the point of pouring quality in to your work only to have the immersion broken by a door that defies the laws of physics.
Another bug reported was if I turned all the lights off in a particular room, including the flashlight. Stand in a particular corner and face a particular way. The light from the next room shines through the wall. This is a great example of not being able to emulate everything a player might do. When play testing it never occurred to me to try this at all. I'm running through testing implemented features but standing in spots and trying a range of scenarios has escaped me. Mostly due to time but also because I don't play the game that way.
This brings me to my point. Coming in to this I didn't expect any surprises out of QA testing. After all, no one knows my game better than I do right? Whilst that may be true, these players know their play styles much better than I do and for that reason. I cannot overstate the value of QA testing. If you're a small developer like me, QA test with as many people as you can. It will save you a ton of trouble down the road.
QA is still going on and 34 bugs have been found and resolved. The game is running much better now than it ever has and I owe that to the QA testers.
