Dealing with feedback
We've had some good and bad feedback from Zex. I thought it might be interesting to write about the types of feedback we generally get for the software we write, not just Zex, and how we deal with it.
We get feedback via multiple sources: emails, forums, download site rating reviews, web-site reviews and magazine reviews.
For a released program very minor things or bug fixes are done nearly straight away but big ideas or major suggestions get put into the melting pot for a major release. These things are relatively easy to plan for although deciding what good ideas go in is more like a process of deciding what to leave out. I think people also know what they expect from a specific program, which makes it easier.
The only possible time for a new program is during testing before release - hence we like to do beta test. At the end of the day we want people to find our programs likeable and able to satisfying a need. This applies to games as much as (if not more so) than applications since people don't expect games to have major updates (I guess because that's what traditionally happens?). It plays havoc with the schedules, of course, but unless you are going to ignore all comments (and why are you doing beta testing?) then you have to allow time to fix things that are wrong.
One way of classifying feedback would be these categories:
General likes and dislikes are good as well although more difficult to deal with. For example: Graphics not good enough? The question is, although it will obviously effect usage/sales, the opposite could be worst - delaying for months to fix the graphics will lose sales and benefit to users now. This is slightly different for a retail commercial game: you get one shot - once it peaks most games disappear (although I guessing development shops have money only for so many months). So you have to try to make a logical decision.
Wide and sweeping statements are almost impossible to deal with: "terrible and primitive" - what do you do with that? It not specific enough - do they mean graphics, game play, crashes, what? In this case it was a review where there is no ability to reply. Another example was a general comment regarding "gameplay" - gameplay is that thing that is everything in games but also nothing. I guess you need to take a long hard look at the game and see whether they are correct and/or if it warrants anything being done because of that comment. Obviously you could decide to delay development and change major sections; even then you might not address the specific issues. It's a tough one. And you are never going to win everybody over - especially in subjective things like games, music, films, etc.
Even if you assume the review is not meant for the authors ... how is it helpful to someone looking for a game to play? One of the problems with the "reviews" on download sites is that they aren't a review. They are usually a one line sweeping statement by someone who doesn't write for a living and you hope (both as an author and as a user looking for something) that enough people comment to get a balanced view. I'm not saying these type of reviews should be scrapped. What other choice is there to decide whether to look at a specific program? There is a lot of software produced and only a small fraction gets a decent review (unless you are either high-volume or you pay for magazine adverts). And even if it gets a review most users will never see that review before downloading.
And there is a big difference between wide sweeping positive and negative feedback. Positive feedback means they may well have used (and in the case of pay-for-software bought) the software. Negative feedback makes you want to (a) give up, (b) ignore it, or (c) try to see what they are getting at (hard to do accurately). The best option - see what they are trying to get at - could easily make you make a wrong decision for the program.
In regard to general feedback there is also something else that happens: you appear to get a lot more very positive and very negative feedback than maybe is representative of the people who have tried the program. This is perhaps natural: people who feel strongly are might likely to go to the effort of writing something? Certainly we've had some very positive feedback. How should this affect our view of the negative?
Conclusion
In our case I guess we will fix what we can and keep trying to get more feedback to decide whether the big things need fixing. We've just released "0.99.3", the third beta release for Mac and the first beta for Windows. It's just an incremental release from last week for Mac (fixes crash bugs) - but hopefully should give us some more feedback before we do anything drastic.
We get feedback via multiple sources: emails, forums, download site rating reviews, web-site reviews and magazine reviews.
For a released program very minor things or bug fixes are done nearly straight away but big ideas or major suggestions get put into the melting pot for a major release. These things are relatively easy to plan for although deciding what good ideas go in is more like a process of deciding what to leave out. I think people also know what they expect from a specific program, which makes it easier.
The only possible time for a new program is during testing before release - hence we like to do beta test. At the end of the day we want people to find our programs likeable and able to satisfying a need. This applies to games as much as (if not more so) than applications since people don't expect games to have major updates (I guess because that's what traditionally happens?). It plays havoc with the schedules, of course, but unless you are going to ignore all comments (and why are you doing beta testing?) then you have to allow time to fix things that are wrong.
One way of classifying feedback would be these categories:
- Bug reports
- General likes and dislikes
- Wide and sweeping statements
General likes and dislikes are good as well although more difficult to deal with. For example: Graphics not good enough? The question is, although it will obviously effect usage/sales, the opposite could be worst - delaying for months to fix the graphics will lose sales and benefit to users now. This is slightly different for a retail commercial game: you get one shot - once it peaks most games disappear (although I guessing development shops have money only for so many months). So you have to try to make a logical decision.
Wide and sweeping statements are almost impossible to deal with: "terrible and primitive" - what do you do with that? It not specific enough - do they mean graphics, game play, crashes, what? In this case it was a review where there is no ability to reply. Another example was a general comment regarding "gameplay" - gameplay is that thing that is everything in games but also nothing. I guess you need to take a long hard look at the game and see whether they are correct and/or if it warrants anything being done because of that comment. Obviously you could decide to delay development and change major sections; even then you might not address the specific issues. It's a tough one. And you are never going to win everybody over - especially in subjective things like games, music, films, etc.
Even if you assume the review is not meant for the authors ... how is it helpful to someone looking for a game to play? One of the problems with the "reviews" on download sites is that they aren't a review. They are usually a one line sweeping statement by someone who doesn't write for a living and you hope (both as an author and as a user looking for something) that enough people comment to get a balanced view. I'm not saying these type of reviews should be scrapped. What other choice is there to decide whether to look at a specific program? There is a lot of software produced and only a small fraction gets a decent review (unless you are either high-volume or you pay for magazine adverts). And even if it gets a review most users will never see that review before downloading.
And there is a big difference between wide sweeping positive and negative feedback. Positive feedback means they may well have used (and in the case of pay-for-software bought) the software. Negative feedback makes you want to (a) give up, (b) ignore it, or (c) try to see what they are getting at (hard to do accurately). The best option - see what they are trying to get at - could easily make you make a wrong decision for the program.
In regard to general feedback there is also something else that happens: you appear to get a lot more very positive and very negative feedback than maybe is representative of the people who have tried the program. This is perhaps natural: people who feel strongly are might likely to go to the effort of writing something? Certainly we've had some very positive feedback. How should this affect our view of the negative?
Conclusion
In our case I guess we will fix what we can and keep trying to get more feedback to decide whether the big things need fixing. We've just released "0.99.3", the third beta release for Mac and the first beta for Windows. It's just an incremental release from last week for Mac (fixes crash bugs) - but hopefully should give us some more feedback before we do anything drastic.
posted by RobZed at 1:09 pm
0 Comments:
Post a Comment
<< Home