Guerrilla Tool Development

I have a weak spot for cool game development tools. Not the IDE, or art or sound tools – I mean the level editors, AI construction tools – those that developers develop specifically for their games. Those that you know could help you multiply your content, and craft your game just a little bit better.

Level editor

Unfortunately, if you work on a small team, developing sophisticated tools like that is pretty much out of the question. That does not mean you have to hardcode everything, though. Here I will give you some ideas for getting tools for your game on a tight budget.

Know your content creation tools inside out

Before you even think about developing customised tools, it is extremely important to know your content-creation tools extremely well – even if you are not the content creator. Often, some game specific functionality can be obtained from a content tool with a little bit of tweaking (there is a section about that below).

As a programmer, you should focus on the following features:

Automation

Many art tools support some kind of batch processing. This is of course very useful for repetitive tasks, but more importantly it is useful to enforce consistency. One of the most common human errors is to perform repetitive tasks incorrectly; automating this process avoids this kind of error altogether.

Data driven design

This goes hand-in-hand with automation. Not only does data-driven design avoid repetition and error, it also makes it a lot faster to change content – either the detail (data) of a specific element, or the design in bulk. Data driven content can also be multiplied with relatively little effort; especially if you design content to be naturally combinatorial (that is – you can mix and match components).

Data-driven techniques are particularly useful for interface art (buttons, icons, etc.), especially since most games do not use custom widgets available through the operating system.

Extensions

The more you can extend a tool – and the easier it is – the more you can customise it for your specific usage. If you can extend an application, you can basically turn it into anything you want. Here are a few questions to ask:

  • Can you traverse the objects in the file?
  • Can access object properties?
  • Can you access primitives (pixel colours, vertex positions, etc.)?
  • Can you read and write files?
  • Can you access external libraries?
  • Can you access the application functions?
  • What tools (brushes, selection tools, etc.) can you emulate?
  • Can you build a GUI for your plug-ins?

Organisation

Layers

Layers

You should know how content can be organised in a particular application. In image editors, you typically have layers; in vector art editors, you have groups, layers, and pages. In 3D apps, you have layers, models, groups, and a variety of other means. You should also know how (or if!) these structures can be named.

Any extra functionality that you implement (through automation, data-driven features or extensions) can take advantage of the organisation or names of the data. For example, putting all objects to be processed in a special layer is much better than forcing someone (maybe yourself) to select 100 objects manually for processing.

Of course, you need to carefully consider the different options available, how it will impact content creation (you do not want to enforce something that the content creators can’t live with – for example, putting all the game textures in a single Photoshop file!), and how easy it will be to change your mind.

Tree

Tree

Naming conventions are good to have, but you should avoid writing tools that rely on names:

  • Names are harder to change than to move stuff from one layer to another.
  • As the functionality you are trying to implement grows, you might end up with names that are Turing complete – and hence very difficult to name correctly: pedestrian-health100-walk3-if-attacked-run-while-hungry-search-food. This info does not belong in a name.
  • Writing code to parse names can become very painful, especially if the naming convention changes during the project (and it will!).

Templates

Templates are useful when no specific data-driven features are available; especially if changing the template also changes the assets based on them. Like data driven design, templates eliminate production monotony and error.

Supported formats and interoperability

You should already know that your tools support formats that will be used by your game or game engine. But you should also know what other formats it supports; in particular, any format that is open enough for you to abuse for some other purpose. The first thing I look for is XML or some other text format. If the tool supports this, you can “reinterpret” an asset as something else very easily. Of course, this can be done with an arbitrary format, but in general you would not want to write the code for that.

You should also be aware of what resources can be shared between programs. For example, Photoshop plug-ins, brushes, etc., can also be used by the Gimp.

Use appropriate file formats

If you implement a tool that spits out game data, the file format you choose have a big impact on the amount of work required. In general, keep these principles in mind:

  • Human-readable formats are great for debugging the tools that create data, or spotting logical creation errors (for example, putting an object below the world plane by accident). It also makes it easy to create quick test data by hand, or to modify existing data quickly (also for testing).
  • Human-readable formats should always support comments. Not so much for documentation but as for rapidly taking out some data temporarily. This is important for testing.
  • Human readable formats should support flexible white space handling. The tool should be able to read lines that start with spaces or tabs, or jumps over empty lines. This allows files to be better organised visually (whether produced by man or machine), and prevents “invisible” problems (such as empty lines that crash the parser).
  • Using custom formats is usually not a good idea. Using existing formats will often save you the work of writing parse code, and will help you to reuse tools across projects.

Property Files

Here is an example of a property file:

level = Jupiter
aliens_count = 5
space_ship = Androxi II

These files are relatively easy to parse, simple to edit (also for non-programmers).

Make sure that your importer does not rely on a specific order – this makes the tool easier to maintain, and helps make it backwards compatible).

Property files are only suitable for flat data. If more structure is required, XML should be used instead.

XML

XML is by no means the best format for data. It tends to be verbose, and does not represent some data structures, such as graphs, well.

However, it is so widely used that it would be a terrible waste not to piggy-back onto existing code and tools:

  • XML parsers are readily available in almost any language. All you have to do is traverse the tree. Moreover, object serialisation in many languages (such as C# and Java) has already been implemented to work with XML. That means, you can read in objects directly from their XML presentations, so if your tools produce XML in these formats, you do not even have to load in class attributes manually.
  • XML is used for a wide variety of file formats. These file formats can readily be read into your game, allowing you to use tools a bit more creatively. For instance, SVG file format (that can be exported from vector art tools) can be used as a level description file.
  • There are many XML editors available, many of them for free.
  • XML can be rendered with formatting that is specified independently of the data. If you implement DTDs for your files, visual inspection can be very powerful.

XML is also editable by non-programmers (after a 5 minute training course!), and the verbosity actually helps document the format.

Scripting Language as a data format

If your game engine supports scripting, it is often better to generate data in the scripting language directly. This means you can use the built-in parser, and, if it is available, also the editor. In addition:

  • It allows for “smart” content, (for example, a property whose value depends on other properties). This is especially useful if the scripting language is also used for other purposes (for example, Python), which allows use of external libraries.
  • It allows for more complicated data structures.

On the downside, it is not suitable for hand editing by non-programmers. Also, smart content is dangerous, especially if it cannot be checked in the tool where it is created. For those reasons, it is best to limit your format to use only the most necessary data presentation features of the language (dictionaries and structs, for example).

Game or Game Engine Language as Data Format

In some cases, you might want to produce code files to be compiled with your engine directly. This is most useful for files that control behaviour, (such as state machines or other AI behaviour), scripted events, procedural content, or graphics routines (such as shaders).

The main advantages of this approach is that it

  • can give a performance benefit (such as in state machine or shaders),
  • gives support for smart content when no scripting is available, and
  • saves writing complicated code for handling an intermediate format (such as for graphs).

It is also a dangerous approach, and can easily become a production bottleneck:

  • It is generally harder to make it work, because of the additional constraints in syntax and names (to avoid name collision with other engine code classes, for instance).
  • Each time the asset changes, it must be recompiled. This can become a problem in bigger projects if it is not dynamically linked.

Don’t be blind-sighted by the “purpose” of content tools

You should always use the right tool for the job…right? If you have lots of time / money, then yes. Otherwise, it can be helpful to use a little imagination and use almost the right tool for the job. The hard part of creating a tool is often the interface. So choose an application that already has the interface, build in some smart logic, and you have a tool to go.

2D vector art tool as a path editor

Paths are used by AI (in racing games, for example) and cameras. They can simplify the intelligence of an agent or camera considerably, but they are a pain to construct, especially if the basic level structure change often.

One approach that works well is to use a vector art tool (such as InkScape) to build paths. You can save files in SVG format (and SVG is XML), so you can easily grab the coordinates of a curve. Of course, you will need to scale and offset the values to make it work in game.

InkScape can be used as a path editor

InkScape can be used as a path editor

2D vector art tool as a level editor

Any tool that allows you to place 2D objects can be used as a level editor; but it is usually only feasible for very simple scenarios. For example, you could use colours from a palette to distinguish different objects, and only use rectangles to represent them. This is probably only viable for a small amount of different objects – if you cannot remember the appropriate colours for all objects, production will grind to a halt; it will be error-prone and painful.

As with the curve editor, you cannot annotate objects with properties. Well, not easily – in simple cases, you might want to use text boxes to put in properties. Again, for objects that have too many properties, this approach will be error-prone and painful.

Spreadsheet program as a level editor

For grid-based games, spread-sheet programmes can make nice tile-placement editors. There are many ways you can do this; here is one:

  • Make cells square – make sure they are large enough for their contents.
  • Use characters to represent different tiles (‘w’ for water, ‘s’ for soil, etc.)
  • Use conditional formatting to get a nice visual feedback.
  • Save the file in some XML format that is easy to parse in from the game or game engine.

You can also use multiple cells per tile. Using four cells, for example, lets you specify the tile, a monster type, the number of monsters that will spawn, and a special object.

A simple spread sheet representation of a game level. Blue is water, orange is sand, grey is rock, and green is grass.

A simple spread sheet representation of a game level. Blue is water, orange is sand, grey is rock, and green is grass.

A more complicated spreadsheet representation of a level. The tiles are as before, now only occupying the top-left corner of each tile. Items have black backgrounds, and mnemonic font colours. The number of enemies is shown (colour-coded) in the right-bottom corner of each tile.

A more complicated spreadsheet representation of a level. The tiles are as before, now only occupying the top-left corner of each tile. Items have black backgrounds, and mnemonic font colours. The number of enemies is shown (colour-coded) in the right-bottom corner of each tile.

A more complicated spreadsheet representation of a level. The tiles are as before, now only occupying the top-left corner of each tile. Items have black backgrounds, and mnemonic font colours. The number of enemies is shown (colour-coded) in the right-bottom corner of each tile.

Normally I would not recommend character codes for any kind of information. However, when building a level with the keyboard in a spreadsheet editor, it is important to type rapidly. As long as you do not have too many types of tiles and other info, this is a sane approach. Just remember to document your codes! (This can be done in the file itself).

You can also include other pieces of level information in the file (for instance, in a specially allocated column (typically the first), multiple levels are easily stored in a single file, and you can copy, paste and drag sections. After getting used to this kind of level editing, you will find other tools cumbersome.

It is easy to go overboard with this method. Be careful to keep it simple. Except for the importer, the hard work is implementing the formatting rules. Make sure that you allow enough flexibility to add rules without needing to redo old ones. If you try to visualise too much information, you will not be able to seeanything!

3D modelling tool as level editor

Most 3D modelling tools have most of the functionality needed for a 3D game level editor. Most importantly, they have all the interface components you will need. It is often desirable to implement you level editor in your game engine itself as this will give the best impression of how the final result will look, and AI, lighting, and physics parameters can be tested immediately. However, implementing 3D controls that work properly can be a challenging task (especially since game controls and 3D build controls have some important distinctions – “flying” towards the region you want to edit can be a very slow way of building 3D worlds.

With a little bit of work, you can export levels directly from the 3D editor. Most sophisticated 3D tools allow you to add additional properties to objects – this can be used to put in game content.

It is imperative to keep assets and level information separate; that is, an asset should be editable without the level file being opened. Do not import models into level files. Instead, you can rather put in linked copies of models (reference models) or representative dummies into the level file.

Another game engine level editor for your game

Your game might be unique, but chances are that it shares many features with some other game – some other game for which a level editor is freely or cheaply available. It’s a good idea to keep your ear on the ground and be on the lookout for free level editors of any type. You will probably have to do some conversion and re-interpretation (for instance, treating “aliens” as “trolls”), but if the interface is good, you will save a considerable amount of time.

Other game creation tools (such as Game Maker and TuDee) are also viable options – even for some 3D games. Again, the strength of this approach is that the interface has already been developed – all you have to do is get the data into the right format.

Computer Algebra Systems for content processing and AI

Computer Algebra Systems such as Octave (and its expensive cousins Matlab and Mathematica) are very useful for math-heavy operations that need not be in your game (for example, training algorithms for neural nets and image filters). They typically support

  • high-level syntax for matrixes and polynomials that enables rapid development and lean code,
  • 2D and 3D graph plotting (which is useful for statistical AI techniques),
  • fat libraries that include functions for filtering, random number generation, sampling, curve fitting, and frequency analysis,
  • some form of GUI development.

Many AI, image, mesh, and sound algorithms have already been implemented on these systems, and are available.

Matlab

Matlab

Conclusion

Good tools need not cost a fortune (in time or money). With some thought, you can put down a pretty sophisticated pipeline. If you have some ideas for getting tools in the spirit of this article, please share it in the comments!

  • Nandrew

    Very nice article, Herman! You’ve actually opened my eyes in quite a few respects, especially with regards to creative sources for level editors (those really ARE quite a pain at times!). It’s particularly nice to see the suggestion for using Game Maker and the like as level editors: I actually did the same thing for one or two Flash projects that I’ve screwed around with, and it really does work. ;)

    • Herman

      Thanks! The Game Maker idea actually comes from one of Danny’s articles for NAG (quite a while ago), I think it was about data driven design. Before then, I never thought of GM as a tool for anything other than GM games.

  • edg3

    Small error on page 1 ‘can you write and write files?’, but I enjoyed the article and found it quite useful

    • Herman

      Hmm, thanks for spotting the typo; I’ll ask Chippit to fix it…

  • Pingback: Guerrilla Tool Development · code-spot

  • Pingback: 50 Tips for Working with Unity (Best Practices) » devmag.org.za

  • http://bit.ly/5bip23 Ritz_ND-44-ZH

    Some additions:

    - instead of XML you can use YAML. it’s also got a wide range of parsers and libs available, but it’s much less verbose and easier to read (in its simplest form, it almost looks like your properties-file example).

    - instead of Octave/MATLAB you can use Python+PyLab/Numpy/Matplotlib+IPython it’s got nearly the same syntax for graph plotting as MATLAB (heavily inspired) combined with the scripting sanity of Python! ;-)

  • Pingback: ??Unity?50??? | ?????

  • Pingback: Unity3D Tips

  • Nahuel Bergamo

    Interesting read, I’ve been wondering about whether the UDK Level editor would make a fine level editing tool for use within my Unity3D project… as it now can export to FBX, maybe I can get that from Unity3D.
    Its in the same spirit as you mention here, I like the suggestions you did, will certainly take this mindset into consideration more often :)
    Cheers!.

  • Pingback: Unity Development Tips