Play-testing and Robot compilers

Zex is currently under play test. It so easy, especially when you have a bugs list, to just fix bugs and think it's done.

Part of the reason we do beta test releases is to get real play-test feedback. But it's critical that we actually use the software we are developing before doing alpha or beta test releases.

Then, after we think it's ok and release the test versions, the next thing is to fix the game-play problems identified. However, this is where the danger can be. You can do theoretical changes to a game and if you don't try them properly from the start of the game then they might easily make the game a lot less playable than before.

Robots

Also recently I've installed the gcc compiler for ARM for a robot I'm building. Just need to find 5 minutes to write some code for it now ... the hardware isn't all that far off. I already had my AVR compiler working.

Re-writing software from the ground up

There was an interesting article on Slashdot (normal warning: evil time-waster, etc) - but not interesting in my opinion for the reasons I think you might think. That is, that rewriting or refactoring is something all software developers face at some point so it should be interesting - but that's not the only thing that's interesting to me. (I'm ignoring the outsourcing and manager suggestion angles totally in this short blog entry.)

First a link to the article:

"Moving a Development Team from C++ to Java?" http://ask.slashdot.org/article.pl?sid=06/05/18/0154213

It's related to a good (old) article by Joel ...

"Things You Should Never Do, Part I" http://www.joelonsoftware.com/articles/fog0000000069.html


My Standard Statement

So what do I believe? Well my standard (simplified) answer is: Re-writing is a very bad idea for anything that is not tiny. Throwing out all your code throws out many hours of design, development, testing (developer, tester, customer), bug fixes, customer requirements, established quality, etc, etc. Refactoring all your code bit by bit is a better option - and I always say don't be afraid of change. Rewrite a 'module' at a time - the worst first. Be careful to change the architecture where necessary with scaffolding where necessary. Of course this will lead to bugs in that changed 'module' but usually that outweighs the bugs you'll get if you are changing a bad module anyway and especially if it's really unmaintainable. I usually follow that up with a link to Joel to give extra weight to the 'don't re-write' argument.

There is another minor point: in companies, it appears a lot of re-write projects can cancelled and the previous code-set get's advanced. Elsewhere they just never get finished. Of course, I have to say that, you have to be very careful this isn't the reason you don't re-write. Avoiding all risks is dangerous as well and avoiding risks is a bad reason, IMHO.


Complications in Reality

However, as it turns out the situation is a lot more complicated than never re-write. Sometimes (as maybe the Joel's Mozilla case might or might not be) it's your only option. Either the code is obsolete (language, API, machine, etc) or doesn't do what you want - not just that it isn't maintainable. I still think you should not take it lightly at all - in my experience usually it's the wrong choice. A couple of warning signs are is if it was written by someone else (or yourself a long time ago) then the effort to learn the code and structure always looks hard. That doesn't mean the effort to re-write won't be longer.

Another good reason to re-write might be if you are doing it for fun and not money. For exmaple, people critise some open-source projects for being rip-offs of other projects. But if they are doing it purely for fun then why shouldn't they re-write? Usually they learn at the same time. However, if they want to use the functionality of the program for something else and/or want to do it in the shortest time - then it may well be the wrong option.

There are other options, for example, sometimes you can develop in parallel another version. Call this a "test version" or a "feasibility study" or the "medium to long term version". You don't stop developing the current version in this model. Both are moving. Of course, this takes double the time and extra money, can leave you with two versions to maintain (especially if you get as far as customer testing). But all routes have risks. It does reduce the damage that stopping the previous version or killing the new version can have.

It appears as if this would take longer - and it probably does. But time is not the only constraint - it's especially good if you are trying to achieve a slightly different goal. Say you want a single general version (code-set) and all you have are 2 different versions, that do different things in detail and approach but are basically similar in the problem they are solving. A superset program might be impossible to achieve via merging - in fact that is very likely the bigger a problem gets. And, in this case, it does save time - eventually. In this case it's because, whilst you have two versions, if you need to need to add a feature to both then it may well be you might need to add it twice with two totally different pieces of code.


Languages are Probably Minor

Taking another point off the slashdot article, which a lot of people mentioned, you can, of course, switch languages (both fast and slowly - there are plenty of ways to patch together two languages whilst the transition happens) but the differences between languages usually don't give enough advantage on their own to produce a better program. Sometimes they lead to a new technique (e.g. OO or application specific languages) which suite the problem domain and really do lead to big advantages or they might allow use of a specific library or framework to add functionality you don't currently have. However, I digress.


Did Joel Choose the Wrong Example?

Someone (kupci) on in slashdot comments that "Joel's article seems to be the defacto standard against rewriting software, but funnily enough, I think he's been proven completely wrong, in hindsight, by the very example he used to prove his point. Think of Mozilla [mozilla.org]? Mozilla originally started from the Netscape codebase, but they got a lot of flack from folks like Joel for rewriting it. " Is this right or wrong?


Conclusion

So where am I leading with you in all these random bits?

When to re-write is a complicated question, in which a lot of questions need to be asked, thinking, looking at the exsting code, judging not just the technical merits but the requirements outside of the code (e.g. business requirements), possible other approaches (e.g. code refactoring) and investigation into the real problems with the code-set need to be looked at.

It's therefore probably too difficult for the knowledgeable slashdot readers (I'm not being sarcastic) to make a good decision because someone can't summarise all the necessary (and quite often subtle) information. Someone will say it's good and someone will say it's bad. They could be right - depending upon the situation.

This is what I find the most interesting!

Just don't

Don't play World of Warcraft.

Yes, I know it's good with lots of stuff to do.

But it eats time and is very addictive. It's worst than reading slashdot. (Of course the slashdot comments eat time AND make you feel bad that you've wasted all that time on complete and utter [[[TRANSMISSION TERMINATED]]]

Zex and other things

Bit of a mini-brain dump here of a random collection of thoughts.

Poorfish (see the Zex forum) posted some really cool and detailed feedback about playing Zex. This is certainly the best gameplay feedback we've had. Thanks again Poorfish. I've also learnt some new Zex tricks!

Zex: It's still my goal to do a full release of Zex. We have a LOT of different game projects on our collective hard drives. Zex is the closest to being finished, hence our multi-month resurrection of this yet 'unpublished' game. We've released various applications and programs both under Lightsoft and elsewhere. But no full game yet; a few game demos but nothing officially 'a full game'. Which is ironic if you understand anything about me, Stu or Lightsoft...

My good friend George has started playing World of Warcraft. This is bad news because he's the guy I generally play games with (for example, we completed the three NWN packs a while back and we are on Diablo 2 and it's expansion pack at the moment). However, WoW is a life swallower (like most MMORPGs) ... very dangerous situation for my programming ;-)

Programming:

Fridays thought: Singletons - are they an overused and abused pattern that is not real OO? (Maybe a blog post on this in the future since I wrote some stuff down on Friday).

Zex: What with me being ill around last weekend and most of the week, and the two weeks before being basically down time for us (various reasons), we are a bit behind on our planned date for the full game release. I'd like to fix at least a few of the things that Poorfish has reported (both recent bugs and enhancements). Additionally there are 44 items (bugs and enhancements) on the bugs list we listed as must fix before 1.00 'production' release.

Oh and the beta time has expired on 0.99.3 (beta 3). Oops.

Great Software

I'd just like to say I think existing Free Software and Open Source Software (and, for that matter, public domain software) is great - really, really great. Sometimes people give it a really hard time, saying things like it's rubbish or poor quality or just a rip-off of commercial software.

However, I just think that they are griping for no reason. They don't have to use it! Of course there are loads of abandoned applications on places like sourceforge and elsewhere that are not finished, and others that are worked by learning coders who write terrible code with bugs everywhere that may never be finished. However, it's not like an infinite number of monkeys that produce the occasional gem. The good ones are well know and work well. Additionally, and of critical importance, many are written by commercial companys as well as professional programmers in their spare time and not just hobbists in bedrooms at night as some suggest. I guess most big projects have some combination of the above.

Additionally the whole 'rip-off commercial software' angle tends to focus on the desktop environments of KDE and Gnome. I'm not going to answer that directly but will ask two things: Is a 'natural' bar on selling, effectively, the same software for eternity all bad? Should we restrict the freedom of people or companies to develop competing software that they decide they should give away for free unless there is a another very compelling additional reason to stop them? On the last point think of two situations: browsers and Free Operating Systems.

This is getting way too deep and philosophical!

Let me re-focus on what I wanted to do and give you a list, in no particular order, of some great Free Software and Open Source software that comes to mind. I'm sure there won't be any surprises and I know there will be many, many things missing that I'll kick myself for later.

• gcc ... the great, the wonderful, compiler collection.
  
• cvs ... maybe showing it's age but in other respects still revolutionary and even with all the known flaws still out-performs some current commercial SCM programs.
• The Linux Kernel
• FreeBSD
• OpenBSD
• NetBSD
  
• OpenSSH
• Apache
• Perl
  
• Python
• cvstrac ... a less known program but no less great ... a web interface to cvs (and others) plus bug tracking plus wiki pages.
  
• Ruby
• Lua
• Firefox
• Mozilla
• Eclipse ... Java based IDE
  
• Dev-C++
• MySQL ... open source database
  
• PHP
• Blender ... great open source 3D design package
• GDB ... the debugger
• OpenOffice
• Tcl
• Nethack
• SDL
• Subversion
  

You might note that there are a lot of programming languages and programming related things on this list. This is not suprising for I'm a programmer! Additionally programmers work on F/OSS so programming related things tend to be done well.

And that's all I've got to say for now.

Dealing with feedback

We've had some good and bad feedback from Zex. I thought it might be interesting to write about the types of feedback we generally get for the software we write, not just Zex, and how we deal with it.

We get feedback via multiple sources: emails, forums, download site rating reviews, web-site reviews and magazine reviews.

For a released program very minor things or bug fixes are done nearly straight away but big ideas or major suggestions get put into the melting pot for a major release. These things are relatively easy to plan for although deciding what good ideas go in is more like a process of deciding what to leave out. I think people also know what they expect from a specific program, which makes it easier.

The only possible time for a new program is during testing before release - hence we like to do beta test. At the end of the day we want people to find our programs likeable and able to satisfying a need. This applies to games as much as (if not more so) than applications since people don't expect games to have major updates (I guess because that's what traditionally happens?). It plays havoc with the schedules, of course, but unless you are going to ignore all comments (and why are you doing beta testing?) then you have to allow time to fix things that are wrong.

One way of classifying feedback would be these categories:

• Bug reports
• General likes and dislikes
• Wide and sweeping statements

Bug reports are easy. You just need to decide whether it's important enough to fix. Not all bugs need fixing especially if they are very minor or might not even be a bug. Very low priority bugs (low priority for your users/customers and you) aren't valuable enough for your users/customers to 'pay' for (either in delay in shipping higher priority bugs or in money terms). This applies to free and open source as much as commercial. It's better to ship than spend forever fixing the unimportant.

General likes and dislikes are good as well although more difficult to deal with. For example: Graphics not good enough? The question is, although it will obviously effect usage/sales, the opposite could be worst - delaying for months to fix the graphics will lose sales and benefit to users now. This is slightly different for a retail commercial game: you get one shot - once it peaks most games disappear (although I guessing development shops have money only for so many months). So you have to try to make a logical decision.

Wide and sweeping statements are almost impossible to deal with: "terrible and primitive" - what do you do with that? It not specific enough - do they mean graphics, game play, crashes, what? In this case it was a review where there is no ability to reply. Another example was a general comment regarding "gameplay" - gameplay is that thing that is everything in games but also nothing. I guess you need to take a long hard look at the game and see whether they are correct and/or if it warrants anything being done because of that comment. Obviously you could decide to delay development and change major sections; even then you might not address the specific issues. It's a tough one. And you are never going to win everybody over - especially in subjective things like games, music, films, etc.

Even if you assume the review is not meant for the authors ... how is it helpful to someone looking for a game to play? One of the problems with the "reviews" on download sites is that they aren't a review. They are usually a one line sweeping statement by someone who doesn't write for a living and you hope (both as an author and as a user looking for something) that enough people comment to get a balanced view. I'm not saying these type of reviews should be scrapped. What other choice is there to decide whether to look at a specific program? There is a lot of software produced and only a small fraction gets a decent review (unless you are either high-volume or you pay for magazine adverts). And even if it gets a review most users will never see that review before downloading.

And there is a big difference between wide sweeping positive and negative feedback. Positive feedback means they may well have used (and in the case of pay-for-software bought) the software. Negative feedback makes you want to (a) give up, (b) ignore it, or (c) try to see what they are getting at (hard to do accurately). The best option - see what they are trying to get at - could easily make you make a wrong decision for the program.

In regard to general feedback there is also something else that happens: you appear to get a lot more very positive and very negative feedback than maybe is representative of the people who have tried the program. This is perhaps natural: people who feel strongly are might likely to go to the effort of writing something? Certainly we've had some very positive feedback. How should this affect our view of the negative?

Conclusion

In our case I guess we will fix what we can and keep trying to get more feedback to decide whether the big things need fixing. We've just released "0.99.3", the third beta release for Mac and the first beta for Windows. It's just an incremental release from last week for Mac (fixes crash bugs) - but hopefully should give us some more feedback before we do anything drastic.

Start of March Update

We got a second Mac beta out on Tuesday. Rather caught up on us. Now are we trying to test the Windows version so we can beta test that as well.

I knew we had a crash problem in the Windows version. I suspect this is a code fault (I just confirmed that it is a bug in all versions). Last night I spent the night trying to get the debugger to work in Dev-C++ (normally the Windows version is compiled on a Mac, but for debugging I can't be bothered setting up the cross-machine gdb session). The problem was that it would use the project folder directory for the run session even though the executable was elsewhere ... which meant the associated files couldn't be found. This definately wasn't happening when I used it before with that project file. However I just couldn't find the setting to fix this. Some nights you just aren't productive. Tonight I'd fixed the problem in about 10 minutes by moving the project - a workaround that entirely solves the problem. Ah well.

In the meantime one of the HWC decoders is broken again. Arggg....

Back to the editor then...

Update Saturday 25-Feb-2006

We've been trying to get through some of the high priority bugs and various game-play changes in order to do a beta release this weekend in time for the previous beta expiring.

One of the changes was to add the game level and player rank to the saved game selector menu. This is pretty easy since there is a tiny interpreter used to read the save game meta-data for the game selector and an object that represents the game file on disk. A change to both of these plus a change to the save routine did the trick. I also added a "was this data present" flag which means that older files are handled (it just doesn't show the missing meta-data).

One of the more difficult bugs that is outstanding on my list is a problem with the call-signs. They occasionally appear as garbage (although I've made the display routine just display '?' for parts that are rubbish). Because the bug is difficult to replicate it's quite a difficult one to track down. Additionally we've had bullets that certainly shouldn't have a call-sign being shown with call-signs in the game.

I've also been looking at the highlight in the file selector menu - it's a text arrow like this --> which is a pretty naff. The main constraint is time to make the new effect. Good effects and ideas are easy - fast ones are rarely good. Additionally I really wish the menu code had been properly made into objects. Whilst the code is neat enough, it's sort of a procedural object hack that doesn't support inheritance and virtual functions which means altering the original code. This type of change also happened in the dungeon game that this came out of ... which meant we replaced functionality. It would have been cleaner and more extendable with proper OO. I guess won't have time now either. Maybe for the future.

Guess I should carry on with that call-sign bug.

Java Paradroid

The original C64 Paradroid was a cool 'plan-view' robot battle game by Andrew Braybrook. The Spectrum conversion called Quazatron was good as well - it was 3d 'isometric' style graphics, and I played it for many hours. Anyhow... back to Paradroid.

Of course, quite a few people have done remakes of Paradroid. However the following one is of note because it's a full 3d game, it's written in Java and it's nice and responsive.

http://www.jpct.net/paradroidz/

Who says you can't write games in Java?

Additionally whilst looking around for links to put in this article, I came across a spectrum port of Quazatron written in Flash.

http://bitshifter.net.nz/flash-games/play-quazatron/

I've seen reasonable (i.e. fast, playable) isometric games in flash before, of course - this one is a really cool Dalek game (although all other BBC Doctor Who games suck):

http://www.bbc.co.uk/doctorwho/games/lastdalek/index.shtml

P.S. The original dev diary by the Andrew Braybrook is here. Nice to read from a legend who actually sounds human!

Internet Servers and Conservative Software Developers

Two subjects not related to each other and both a bit of a rant.

Internet Servers

Firstly, as some people might know, the people who arrange the hosting lightsoft.co.uk disappeared a couple of weeks back. They were a small company who had moved the servers around a bit, but originally were cheap compared to other web providers and we'd been running with them for many years without a hitch.

So what happened from our point of view? Well, the web server that hosts out pages went off line and email stopped working. It took almost a week to get lightsoft.co.uk back up and running. The process was basically try to contact them, get a new provider, contact the DNS agency to change over, wait for them to try to contact the old service provider, etc.

The web pages going off-line is bad for a tiny shareware company (for any company?). Additionally, since we released the Zex beta we were waiting for feedback and it had just started coming in on the forum. Practically all that information was lost - since we don't do daily backups. We did have a backup that wasn't very old luckily. In the interim we re-directed the product pages on Versiontracker and MacUpdate to the new provider - but only after we'd confirmed in our own mind the old one was dead and got a new one.

If the web pages going offline wasn't bad enough we also lost all lightsoft email for a week. From a personal point of view this was as bad as losing the Zex feedback or sales since ALL of my personal email comes in via lightsoft (to avoid problems if I have to change ISPs). This includes not only customer support emails but mailing lists, friends and family but internet shopping and contacts for various technical matters.

The moral of this story? I'm not sure. I guess in our case we've gone with a big provider who we know does a good job.

Conservative Software Developers

Doing lots of software I get to work both on my own, with Stu and with larger teams. Both myself and Stu are technophiles. We investigate new technologies, learn new languages, try out new techniques. Contrary to my previous post (regarding C++) this can be things that are not necessarily popular but might be powerful. Of course you don't know whether something is powerful for you until you try it yourself.

Sometimes I have to temper my desire for improvement in the knowledge that constant change doesn't ship. This can be settling with a good enough language, avoiding restructuring bad code or just doing the minimum to ship. This balance is hard to strike but is critical if you are not going to end up never shipping or shipping product that won't be useful in the future. Both ends are bad.

This leads me to my problem; Software developers are meant to be into new stuff. In my experience this isn't true - software developers, for the most part, are not computer geeks. They tend to be people paid to do a job and even the ones who are technical tend to be very conservative in their choice of technology.

This has really being irritating me for the past few months, not because being cautious is a bad thing - in fact avoiding too much change is critical to avoid wasting vast amounts of money for your employer. But there is the opposite extreme - not changing at all.

Thing I tend to see:

Not sure about C++ ... lots more space, confusing to developers, etc. C can do everything anyway.
I don't like OO - might take lots more space, be slower, etc.
I don't like (automatic) merge policy version control, seen problems (manual merges).
I don't believe in UML - don't believe there are any advantages.

Now the problem is that these people demand proof that the techniques or tools will give improvement. As far as I can see there is no magic metric that will give this. But it feels to me, and many others, that some of these things will help improve things like reliability, ability to change code and maintainability. Of course, taken to extremes they can be damaging. But generally can they be so bad if so many (clever) people use them?

Arguments like "well we can do any change you want in this C code" don't mean it couldn't be lots more maintainable. Deciding what to improve is hard, of course. Some changes won't have the desired effect and they may have a negative effect. But it's critical to consider how things could be done better and actually try out some of those techniques lest someone make you irrelevant. Also, the customers for software want quick changes as well as other things - the driver for fast changes as a customer need is much harder to measure. It doesn't mean that it's not important.

As an example of types of change that could be done - migrating from C to C++ is great because you can test the water of OO whilst mostly writing C in C++ compiler is entirely possible. Hence you can test whether it's an advantage (or at least not a disadvantage) and learn as you go.

Luckily with the right approach you can encourage change. It's slow and hard work. And the one advantage with Software Developers: they work with technology and generally have a logical mind. If you ask if it's worth trying to improve software quality or maintainability long term they will very seldom disagree.