Design Bugs

A couple of weeks ago I promised to talk about design bugs.

Simply put, a design bug is an error in thinking about how a problem should be solved. This is different from coding bugs, where the solution is basically correct but isn't written properly.

Here is a real example: Mantis issue 9632 (fixed for 3.5.0). The XML Import feature was designed to use the API Views. This lets xTuple ERP make sure the data are consistent with each other and with xTuple ERP's data integrity rules. So what's wrong with that? The design expected to find all API Views in a database schema called 'api'. This works for views in the core product. It fails as soon as somebody develops a package with its own tables and import API. This extension to the import API belongs with the rest of the package, but the XML import only looked in the core 'api' schema.

The nasty bits with design bugs are that they are often hard to prevent, easy to find after it's too late, and usually hard to fix.

What makes design bugs hard to prevent? Most people think about what exists – it's easy to look at a specification and see things you don't like in it. As a user of xTuple ERP, you should get those issues get addressed1 before the spec is finalized. It's much harder to think about what's missing. In some cases the thing that's missing is an unknown future. In the case of the XML Import dependence on the 'api' schema, we didn't think of the potential interaction of packages with data import, nor of the complexity of the packages that would get developed - scripting didn't even exist in the product yet, so there was nothing to consider. However, sufficiently creative thinking could have spotted this flaw as an unnecessary design assumption.

What makes design bugs “easy to find after it's too late?” It's obvious once you use the software that something's not quite right.2 As soon as someone tried to import data for a complicated extension package for xTuple ERP, the hard-coded 'api' schema became obvious. This design bug was easy to fix but the package developer dependent on it has to wait for another release cycle to get it.

This leads to the third nasty bit – 'hard to fix.' Removing the schema limitation was easy but there is now a compatibility problem. In 3.4.0 and earlier, the hard-coded schema in xTuple ERP means that the package's import views have to be put in the core's 'api' schema. In 3.5.0 and later, these views can and should be put in the package schema. The package developer now has to manage upgrades to deal with this. The new version is backwards compatible, giving the developer time, but the package's views should still be moved.

Most design bugs are much more difficult to fix than this one. Why? Because developers start making assumptions as soon as we write a line of code. We have to or we would never finish anything. Changing assumptions is hard, both for psychological reasons and practical reasons. If the original developer tries to fix things, s/he already 'knows' what was going on and often can't recognize the assumptions s/he made. If a different developer tries to fix things, s/he often doesn't know all of the places in the code that need to change.

Therefore, the best time to fix a design bug is before anyone has written a line of code. If you are reading a feature specification, read everything and don't assume anything! If there is a Delete button in the spec, make sure the spec includes a sentence about warning before deleting. If you expect particular columns to show on a report, make sure those columns are listed in the spec. As painful as it is, read the table descriptions and make sure all of the data you need get stored somewhere. Think through as much as possible how you would use the new feature step by step and check that each aspect is described. Run the feature against a test database as soon as it's released, whether in alpha or beta or final form. The sooner you try it, the more likely we all are to get the feature implemented well.

In many cultures asking questions is considered rude. When defining how software should behave, neither user nor developer can afford the luxury of avoiding questions. Please don't be offended when a developer asks question after question after question. Our job is to understand and give you what you need. We need to ask questions to find out what those needs are.