You may have heard the term “agile development”. This blog entry will discuss this development methodology. A quick visit to Wikipedia.com tells us that “agile software development” is a methodology which “chooses to do things in small increments with minimal planning, rather than long-term planning”, “team composition in an agile project is usually cross-functional and self-organizing without consideration for any existing corporate hierarchy or the corporate roles of team members”; it “emphasizes face-to-face communication over written documents. Most agile teams are located in a single open office to facilitate such communication.”

Agile methodology is naturally juxtaposed to the classical “waterfall” method, which anyone worth their salt in the development world is familiar with.
As someone who had a chance to work in both environments, I want to mull over the pros and cons of each, and then suggest something else.

Let’s go over the two methodologies briefly. As the author, I allow for my own interpretation of some concepts. Also, I use “agile methodology” here instead of naming a specific one (Scrum, Extreme Programming, etc) because for our purposes here, the differences aren’t that important.

What is waterfall methodology?
- The team is comprised of people with well-defined roles and responsibilities
- The team has a formal leader, informal leader and a “client”. The formal leader is the manager, informal leader is the tech lead, and “client” is the product manager.
- The application is first conceptually born outside of the team then starts “waterfalling” from team member to team member - functional design, wireframes, visual design, tech specs, system architecture, development, qa and finally production
- Every step is formally documented, and the written documentation is the “bloodstream”.

What is agile methodology ?
- The team is comprised of people whose roles and responsibility are fluid and interchangeable
- The team has only informal leader[s] and this can be different people depending on the stage of development
- The application development more resembles beating down an elephant- you kick with what you have at hand at the piece closest to you at the moment. (Please note, no actual elephants were harmed in the writing of this blog or any code I’ve ever written).
- The “client” even though intrinsically present (some form of product management still exists), does not have a say on the order of things implemented
- Documentation isn’t maintained in favor of performance; verbal communication is the “bloodstream”

Or, to loosely summarize, the waterfall model is predictable bureaucracy, while agile methodology is creative chaos. Which one is better ?

From my experience, having been a technical team member in both models, as well as both formal and informal leader in waterfall model, I can see value in both. I also see inherent problems in both of them, and while the problems of a classical development model are commonly discussed and became one of those “everybody knows” kind of thing, issues of agile methods aren’t so apparent.

For anyone with substantial experience in software development, especially those who have managed development teams, the achilles’s heel of classical methodology is its behemoth-like dexterity. Which isn’t very “agile.” It takes so many developer-hours of application conceptualization, design and architecture, that, if by the time the developers are cranking at full speed and someone wants to change something, it comes with excruciating pain. The entire wheel has to be turned, the waterfall needs to be fed with updated specs, and the updated design/functionality/feature has to trickle down all the way again. The same thing applies to updates, upgrades, patches and so on.

Its strength ? Predictability and repeatability. Just two words, but they mean a lot for any company betting on its intellectual property in form of software. I know what and how fast my team can deliver. I know I can replace any team member because the process is well documented and I know where to go for information when new people are added. I know the same process can be duplicated with similar results.

Now let’s take a look at the agile methodology’s strength and weaknesses. An Agile team is very….well, agile ! The team can quickly change direction, can quickly adopt new features. Delivering bug fixes is a breeze. I know for a fact that an agile team can deliver a production patch in one day, in the morning a problem gets reported, by end of the day a patch is finished, tested and ready for deployment. That’s pretty much it - the speed and quick turnaround is why managers even consider this development model.

On the flip side however, there are a multitude of problems. When I recently joined an Agile team already underway with the project, the first problem I faced was lack of any information about the system I was working on. There was no architecture documentation, no diagrams, and the only way to learn something was by “breaking in”. Every team member had only a fragment of the overall knowledge, and even though everyone sits and works in one big room without cubicle walls, you can’t easily find the right person to ask because there’s no such thing as a “knowledge directory”. You need to know which fragments exist to begin with, and who owns them. The problem is exacerbated by the fact that some team members were long gone, which renders some parts of the system completely undocumented, even by word of mouth.

After some time I got used to running around and asking everybody about everything. Such fishing would most of the time yield the necessary information, although at the cost of productive time on my part and the parts of the people I was asking. By the time my “break-in” process was mostly over and I was able to find my way around the application, with only about 30 minutes a day dedicated to “fishing”. I discovered the most gruesome toll imposed by such ad hoc development methodology: code duplication. Think about it for second. If discovering what the system already has and how it works is such a pain, wouldn’t it be easier sometimes to just right your own little utility procedure? Of course it would, and that is exactly what happened. Every developer had his own little utility toolbox, the size of which was inversely proportional to the number of years a team member was with the team. The magnitude of the problem turned out to be immense. There was barely any part of the system not affected by duplication. In some cases, adaptors were quickly written to pass data between different versions of essentially the same code written at various times. And the cause of all this was lack of something we all usually see as a nuisance to our development creativity: code documentation.

This was a very powerful lesson for me. Some time ago I led and managed a team of 12 developers, and the company standards dictated up-to-date documentation, maintained in parallel to the development process. I even had a dedicated technical writer, whose job was to keep such documentation in good order, and she demanded updates from me on a daily basis. Today I value documentation for what it really is: an essential part of any application.

Stay tuned, and in the following blog entries I will cover how we here at Naviscent manage to get the best of the both worlds.