Like King of Dragon Pass, Six Ages is a large, complex game. That means that during development, there will be a lot of bugs.
The best way to find them is to start testing early. I like to start QA as early as possible, once there’s a runnable game. For example, we can start the process of making sure every interactive scene works.
We have a really good QA tester, but like KoDP, there are a lot of scenes, with a lot of branches. One of the process improvements over KoDP is that we have a way to brute force test every response of every scene. This is only a supplement to human testing. It can make sure nothing crashes, but can’t find typos or situations where the wrong person is mentioned. But it does mean we don’t waste QA’s time with testing something that isn’t fully implemented.
Some bugs have patterns. If one scene has a problem after the player has made a major story choice, it’s likely that others will too. I usually track these in our bug database as a “sweep,” meaning once all the scenes are coded, we will have to search the code for possible occurrences. We still report fix bugs as we go, and when this situation came up again in a scene, I decided to try out an idea: change the brute-force testing to set up the story situation, and then run the brute-force test. Bingo! Ten failures. Or, ten places that a person doesn’t have to either read code and see what’s going on (in a code sweep), or write up a bug report (manual testing).
Last night I got a bug report about a scene failing when there was no chief. So I decided to do another brute-force test. This found a lot of problems in scenes that had not yet been tested.
Unfortunately, the brute-force testing of every branch of every scene takes about two minutes (and getting worse, as we code more scenes). So while I probably should run each brute-force variant, for now I’m relegating them to occasional situations. But I’m definitely going to look for more places to do this.