Saturday, 5 November 2011

Root Causes of Inflexible Software

Early in my career, I worked for a telecommunication company whose software system had gotten so rigid that instead of features being driven by market pressures, the engineers would tell marketing what could be done.  Most software systems are not this inflexible, but most organizations are not satisfied with the flexibility of their software or the speed at which software can be enhanced and modified.

Some software systems if made tangible would look like the Winchester House, where 38 years of continual building created a house where windows ended up on interior walls and staircases were built that went nowhere.

The inconvenient truth behind inflexible software is that it does not do what you want and that it takes an eternity to get enhancements. You are also stating that fixing it would either 1:) cost too much, 2) take too long, or 3) both.  So ultimately you live with  suboptimal software and make minor changes and wince every time you are asked for a feature that the software should have but that you can not deliver.

Did you ever wonder how your software system got to that state?

Well if you are not getting what you want from your software system then either: 1) the requirements were incorrect when you started, and/or 2) the engineers did not understand the requirements.  Most software projects start off with a formal or informal requirements gathering process with subject matter experts and/or business analysts who write down the requirements in some kind of document.

The nature of every software project is that the requirements are incomplete and inconsistent.  Incomplete because it is difficult to figure out all the details for a software system, especially when you don't allocate enough time for requirements analysis.  Inconsistent because all the sources of information to create the system will come from different view points and represent the needs of the user requesting features.  However, compromise between all the different points of view needs to happen before software is created  to build a practical system.

Good architects resolve incomplete and inconsistent requirements and make sure they are complete enough before starting projects.  'Complete enough', because no one is willing to wait or pay for complete requirements.  Good architects can make bad decisions when they are not given enough time or have bad requirements.  However, some architects are inexperienced and will start to work on the architecture before they realize how incomplete the requirements really are.

This is like telling an architect to build you a two story house, and once the complete informing the architect that you now need to add an additional 10 stories.  Architectural decisions once converted into code become part of an inflexible framework that pushes the software away from what you need.

To make matters worse, time pressures will cause developers to start building out large amounts of code on the suboptimal architecture and will lead to suboptimal code.  Inconsistent requirements will become apparent once you have a version with enough functionality, at this point, missing requirements become clear, which will increase schedule pressure; and inconsistencies with known requirements will require refactoring and recoding.  Cleaning up bad requirements in the middle of a release leads to endless meetings and fire fighting.

Changes to the requirements often require a changes to the architecture. But odds are you are that your delivery date is in jeopardy due to resolving missing and inconsistent requirements.  You may point out that refactoring the architecture is the fastest and most reliable way to end up with quality software, management will insist that you patch what you have and keep going.  Moving forward without major refactoring would be the visual equivalent of putting plumbing into a building that had not been designed for plumbing.  Adding key architecture to software after it is complete will be the source of major defects for years to come.

The extent to which the architecture is inconsistent with the actual requirements will dictate the amount of time and effort necessary to bring the architecture into alignment with the requirements.  The "keep moving" pressure from management may ultimately ruin the architecture and prevent the software from delivering the benefits of the original vision.

If too many critical architectural elements are missing (plumbing, electricity, central heating, etc) then the time and cost of putting these elements in will be prohibitive and the project will fail.  If enough elements are in place, then victory is declared but you will be working with an inflexible application for a long time.

Therefore, one of the root causes behind inflexible software is starting a software project before the requirements are complete and consistent enough.  If you are forced to start building with incomplete and inconsistent requirements then you will end up with insufficient core architecture.  Insufficient architecture will lead will lead to failed projects and/or inflexible software that you will curse for years.