Ganymede was posted for opt-in testing last week and since then I’ve posted seven patches, an above-average number. I don’t think Ganymede was a particularly buggy build, but it included a fair amount of new mid and late game content. As a result, player feedback was spread over a longer period of time. There was one bug that made progressing the story near impossible for one player which I didn’t learn about until Monday.
But overall the feedback on Ganymede has been very positive. I had originally stated a goal of making Ganymede the default within 10 days (June 3rd), but due to the number of late patches and the fact that I will be working offline Sun-Tues, I’ve decided to delay 4 days (to the 7th) to give a little more time for the opt-in testers to uncover any significant issues. Players who want to start it before then can always opt-in.
When not investigating issues and deploying patches, I spend the first half of this week revisiting the game’s analytics to see if I could answer a difficult question: where are players getting stuck/frustrated that I don’t hear about. Analyzing the data shows that players “drop out” at a relatively high rate which then falls rapidly over the 30-60 minutes or so, then reaches a low constant rate. This isn’t necessarily a problem: some players may decide this game isn’t for them, or that the game is for them and they want to wait until full release. There’s also some chance players will stop playing at any point for reasons unrelated to the game.
But if out of the players who play for at least three hours, for example, some disproportionate percentage drop out between two particular critical mission nodes, I should investigate whether there’s an issue there.
My solution to obtain this information was to identify around 50 pairs of mission nodes that I know players must hit, calculate the median time players took between them, and then calculate the per minute drop out rate (Drop_Out_Rate = 1 – Power(Percent_Complete, 1 / Median_Minutes)).
If a node pair has a players dropping out at a rate of 0.5% per minute, while comparable node pairs are dropping out at a rate of 0.2% per minute, then it’s probably worth investigating whether there’s some aspect of the mission that players might miss. As I’ve mentioned elsewhere, there are puzzles and secrets to uncover in the game in side-missions, but the goal is for these not block the main story for a significant amount of time. While progression on the main story is sometimes through organic exploration, if players do not find something critical to advance the story, the game should guide them in the right direction in a reasonable amount of time.
While analytics can help me identify possible areas, I rely primarily on players to tell me where they are having trouble. Given that all the missions are made by me, it is often difficult for me to see what might not be obvious.
Until Wednesday (fingers crossed)!
— Kevin