I’ve always disliked the way MMORPGs are tested. It has always felt like more of a free sneak peak at the game than a real Beta Test to me when a company basically gives you full access to the game 24/7. In my mind with that system very little testing occurs and more “zomG we gots into the game before others so now we know how to be leet” happens as guilds form and prepare to be super uber informed. There are testers (myself included) who would always send in well thought out feedback on the content and truly try to find bugs and perfect the game before release. I’m pleased to see that Mythic has decided to take a more direct and efficient way of testing their systems.
Instead of bringing up the Beta server and leaving it up 24/7 for free play, we’ll be moving forward with a series of focused Beta events set around a specific concept or two. These events will last anywhere from 3-10 days and will enable us to gather specific, detailed feedback from our Beta Community in regards to specific features and locations. We’ll be bringing the server down between each event for a day or two in order to prepare for the next focused event. This will also give us time to process the feedback we receive and incorporate it in future events.
Now that’s a beta test. Players who are chosen to participate in this phase of beta will be given specific parts of the game to focus on and they’re not set loose in the world to “play” the game. Instead they’re being directed to focal points where the players will have no option but to test what the devs want them to test. Taking the servers down gives the beta testers less attachment to their characters which is good – these aren’t your characters you should be attached to, these are your vessels for testing the game. I’m also positive that they’ll get much more feedback from this method of testing because it will put the testers into the testing mindset much more than usual.
Hopefully we’ll see more of this type of testing in the future. Open testing definitely has its place in the beta testing process but definitely not when it’s crunch time.
Open testing makes all the sense in the last month or two of beta, when the game “should” be content complete and general polish is being worked on. So yeah, Mythic’s definitely making a smart move by “chunking” the beta test together to focus on certain things.
LotRO sort of went this way because they released content in bits and enabled auto-leveling when later stages needed testing. I think everyone, love or hate it, can say that game was finely tuned on release because of choices like this.
I think you’ll see more of this type of beta testing in the future as the lessons of previous games become more internalized by the dev community. The days of rushing a game to live with rampant bugs, and having the gamer community just jump right in and pay anyway are slowly going away.
There are too many options out there to run the risk that a similiar game will release with more polish…and steal your thunder.
“Potential” used to be a selling point, but there have been one too many games sold on “potential” that didn’t live up to it…not that I would bring Vanguard up.
That would just be a cheap shot. 🙂
You’re right, this IS a beta test. Leave the servers open for the “Stress” tests.
I was a tester for EA for 5+ years. With each build we received, we were given specific focal points to test. While it wasn’t as much fun testing in this way, it certainly made the end-product a heck of a lot better.
[…] Keen at Keen and Graev’s Gaming Blog recently posted about the latest snippet of beta news from Warhammer Online that basically says the same thing. They are taking the approach of short testing periods with focused goals for each session. This is the best way to involve the players in mass testing. […]
[…] Richard Garriott said, “And the people who did participate in the beta, we’ve had to go back to and say ‘look, look, we promise: we know it wasn’t fun two months ago, but we fixed all that. Really, come try it again.’ We’ve had to go out and develop free programs to invite those people back for free before they go buy it. So the beta process, which we used to think of as a QA process, is really a marketing process.â€Â This goes back to what I’ve been saying. Beta testing isn’t about playing the game. […]