Software projects that must meet stringent development time constraints and continuously changing requirements must achieve agility in their designs and implementations in order to address and accommodate these difficult challenges. A software system that can be implemented quickly and adapt rapidly to meet changing requirements is said to be an "agile" system.

There are several elements that are necessary to achieving an agile software system.

• Identification of the key architectural considerations at the outset of the project (inception)

• Scalable analysis, design, implementation, and regression testing

• Consistent, repeatable structuring to address recurring technical issues (layering, collaborations, delegation, etc.)

• Templatized code body

• Generalized infrastructure for recurring logic

• Strong typing

One of the key elements in developing agile systems is to leverage design patterns in a number of different manners. First, design patterns and programming idioms can be utilized to mitigate project risks by applying proven techniques to resolve technical problems that must be addressed by the software system. The range of problems that can be addressed spans from architectural layering schemes to component configuration to intra-component structuring to resource management (memory, synchronization objects, files, etc.), exception management, and reference counting. Second, design patterns can be templatized and used in a boiler-plate or code generation based approach to rapidly design, develop, and test large segments of the software structuring and templatized logic. Third, design patterns and programming idioms can facilitate rapid development due to the sound, reliable code body that incorporates the best practices that are repeated throughout the code body consistently. Developers do not need to waste time troubleshooting the same problem over and over again in different segments of the same software system. This redoubling of efforts across large projects, as well as the totally avoidable waste of time incurred by developers piddling with problems whose solutions can be automated, both have an exorbinant toll on project schedules, project development costs, and software reliability. There is not much that can be done to avoid the development costs for developing business logic that is unique within a system, but all of the infrastructure that can be factored from this business logic is an entirely different matter!

Good developers know the value of using templatized code. It means that they write the code once, get it working once, and then reap the benefits of these efforts from that point forward. For example, consider the simple iteration of a collection object, such as an array. The original code might look something like this:

Through the use of the iterator pattern, this code can be improved so that the business logic is made independent of the type of collection used by the logic. What if, at some point during development it is decided that a linked list is a better suited solution for managing the collection than the original array?

It is easy to see that this code is easily templatized and that this code can be copied and modified with very little effort to meet the needs of iterating many different types of collections, such as arrays, linked lists, maps, etc. And, this code can be easily modified to conduct different processing on the collection elements that are iterated. So, given the rapid development that iterators offer, why would anybody use any other method for iterating collections? Maybe in a very tight loop that iterates millions of elements under tight time constraints there might be some argument for the inefficiencies incurred by the overhead of accessing the collection via the iterator. But, experienced developers know this is argument is rarely valid given the GHz based processors of todays computing world. Maybe it could be a little faster, but fast enough is all it needs to be, not as fast as is possible. Faster development times are usually more important than code that operates as fast as possible. A project that doesn't meet its development deadlines might never see real operation in the field. And, this is a key realization in adopting an agile approach. Speed of development is king! Worry about optimizing performance later when real performance problems have actually been identified via real measurements. The one exception to this is the areas of development that are known to be governed by strict time constraints. These cases typically constitute less than one percent of the code base in non-scientific applications, specifically business systems. And, even in those cases it is usually a very small segment of the code body that is responsible for the performance degradation. Typically the performance gains are made by changing some strategy in the manner that some particular algorithm is implemented or the manner in which some database technology is utilized, and has little to do with programming style issues.

If you are going to build a software system that has the best performance characteristics possible, you should recognize that the software system that you are building is analogous to an automotive manufacturer that is building a customized Ferari. You should ask yourself, is this what I really need? Or, is what you really need closer to a Mustang that has decent performance, low cost, and fast, reliable assembly line production?

Returning to the iterator example above, though this example is already fairly templatized, it is not good enough for general templatization! Consider the following application of this template where a nested loop is required:

This boiler-plated version of the template does not quite work out because of the naming collision of the iterator naming for the inner and outer loops. It is clear that the variable name "rIterator" is too generic and does not work out in the context of nested loops. Therefore, this naming practice is not the best practice. The best practice is the practice that works in both the context of a nested loop and an isolated loop.

So it is apparent that the naming practice that best templatizes the template is the naming practice that provides more specialization in the variable naming. The specialization can be easily changed with a single search and replace operation when boiler-plating the template, or with a single substitution when code generation is utilized to templatize the code.

The resulting template is the fully-templatized revision of the template code:

Though this form is the best practice because it works in more contexts, it could be argued that the previous templatized form is adequate for most purposes and that it is easily changeable in the exceptional cases.

Taking the templatization a step further in C++, a template could even be devised that made this code truly templatized, as in C++ templates. The template macro could be devised to use this interface to expand to the above code:

The substitution required to instantiate the template for a single instance of an isolated iterator (non-nested) would look like this:

Now that is code that is very streamlined, factored, and has been templatized to the nth degree! But, this template won't work for every case, though it may address a large number of simple collection iteration cases quite adequately. The code for templatizing this code is not trivial because naming substitutions must be made in the template for the class type and the method name. There is a sort of complexity involved in this alone. And, the debuggability of the code may suffer slightly because the code is not as easily visible, though a well written template implementation will help ease any problems with debugability.

Improvements to this template may become more apparent as new applications of it become apparent. The first obvious improvement is to improve the process method portion of the template to allow a more extensive signature for the method. But, I'll leave that as an excursion into the expert domain of C++ templatizing semantics.

Also, it should be noted that for Java, the templatizing would end at the boiler-plated code because template substitutions are not allowed by the Java programming language. This could be considered a short-coming or relief from some of the insidious templatization that occurs in some C++ class libraries and code generation that makes code difficult to debug. But, templatizing some highly repeatable aspects of an applications code body can be quite advantageous. Consider the following code:

This try-catch block might repeat in a large number of methods throughout the code body. By templatizing this code, sweeping changes to the exception management logic can easily be made across the entire code body from a single point in the code, the place where the exception macros are defined. This is agility! The new code might look like this:

The macro also reduces the source code body in a manner that makes it more easy to read. There also might be different macros for the different types of exception management that might be needed for different cases of exception management. And, maybe not every case will be able to make use of the exception catching macro, but if it addresses any significant percentage of the cases, then it is well worthwhile. In practical experience, this type of exception macro has proven to be invaluable for all the reasons described here and more!

Reiterating some of the important points relating to these examples, in the case the reader is at this point off on other tangential thoughts, the templatizing of recurring logic facilitates rapid development because the developer leverages code that has already been developed, tested, and validated. This is an important point because the developer does not have to write code from scratch that needs to be debugged, tested, and validated as original code does. Also, the templatizing of these types of programming idioms relieves the average developers from being concerned with these tedious details. The best developers on the project can develop the idiom templates once and then the entire team can take advantage of their expertise!

And, this discussion has only covered programming idioms thus far. When the discussion is extended to consider design patterns and how they leverage the same types of principles to the realization of designs in the structuring of the software implementation, the value grows by an order of magnitude. It isn't just little snippets of code that are templatized, but entire collaborations and structuring within and between classes that play roles in patterns.

It should be pointed out, if not already obvious, that one of the constraints imposed on this approach is to always maintain strong typing. If strong typing is compromised by either declarative mechanisms, meta-data mechanisms, or insidious casting schemes rather than templatizing, the value of the reuse is tainted by the deferral of type checking from compile time to run-time and the potential (and really inevitable) introduction of otherwise avoidable run-time errors into a software system. If you don't need to downcast, then don't! It's as simple as that. If speed of development is king in the agile paradigm, then strong typing is next in line of succession to the throne because systems that have run-time bugs are not going to get developed quickly. Developers will be wasting time tracking down all of the run-time errors due to weak typing. This is not scalable development. Strong typing eliminates run-time problems before they ever have the chance to manifest. So, if you think you are doing agile development with declarative programming constructs, this is a sort of oxymoron. The fastest path is the path that eliminates problems before they ever happen.

When agile development is discussed, software documentation is typically brought into the discussion. Some "agile" developers attempt to evade the responsibilities of creating documentation for their software under the guise of the agile manifesto. But, a certain level of documentation helps to accelerate development. Documentation that concisely represents key design issues using UML can help to clearly convey important information to developers that will enhance their understanding of the software system and the manner in which it is to achieve its objectives. Using documentation to normalize the development team's understanding of the design is critical to rapid development. Of course, it helps if all of the developers already know and understand the design patterns in play. But, the point is that documentation should not be removed from the project plan for agile projects. The level of documentation should be adjusted to minimize the inertia of the project to design changes.

And, whatever you do, don't eliminate analysis from the project plan for an agile project either. Analysis takes place during the course of the project at some point in time whether it is formally recognized or not. Analysis has to be conducted in order to identify what problems the system must address. It is best to identify the key system considerations as early in the project as possible. The law of 10s says that each phase of development (analysis, design, implementation, test, etc.) that a key requirement is not discovered or is wrong, the cost of repairing it or fitting it in later is a factor of 10 for each phase that development proceeds until it is identified correctly. Of course, different requirements impact the software system at different levels of abstraction and inherently have different degrees of impact, so depending on the particular requirement, the impact could be in the best case rather isolated or in the worst case systemic.

Bridges and automobiles are built rapidly with engineering practices that eliminate problems before they have the chance to manifest based upon experience gained by previous engineering efforts. Software engineering is no different in this regard. Agile development is dependent upon a combination of pre-emptive measures, rigorous, concise analysis and design that yield an accurate analysis model and a durable design model, application of suitable, templatized design patterns and programming idioms, and automated regression testing. Throw in some UML-based code generation to accelerate the whole process and now you are really getting fast. If you are not doing these things, you are not really agile. Hopefully, if you are a manager, your "agile" developers are not really just chickens running around with their heads cut off! This may be a form of agility, but probably not the form you were hoping for... There is a difference between moving really fast and moving really fast in the right direction.

Copyright 2001 - 2003 Christopher Ruel. All Rights Reserved. www.ChrisRuel.net