Waterfall is for Losers?

Perhaps that is a bit harsh, but in the question of whether Agile or Waterfall is best, the reality is that Agile ends up being multiple small waterfall cycles.  So you end up doing Waterfall in pieces.

All projects are composed of a set of tasks, typically executed by multiple people.  Within your set of tasks, some are dependent on others.  In particular we can never get away from the fact that to implement any single requirement, we need to analyse the requirement (what), design the solution (how), code the solution (execute), and test the result.

So any iterative and incremental project, i.e. Agile, Scrum, RAD, XP, etc.  Is really a series of small waterfalls.  The difference is that between the waterfalls we have sandwiched requirements, analysis, design, and coding which gives us the ability to change direction when we need to.

Note: If you never need to change direction (which is not often :-)) then Waterfall is a viable option.  Waterfall makes the most sense in projects where there is little to no requirements or technical uncertainty^*, e.g. reimplementing a system with experienced developers in a new technology where the use cases for the previous system are not changing.

The typical waterfall project assumes the above picture, that all requirements can be done then all design all coding and then all testing.  It is based on a 1960's factory mentality that code can be assembled like a car on an assembly line.

Waterfall only works if the amount of rework in any phase is immaterial and does not materially affect the length of the project.  Typically this is not true.  In the presence of requirements uncertainty, requirements need to be revisited many times.  Often, missing or inconsistent requirements will cause scope to change requiring change requests in the process.

All change requests will affect the project schedule.

In the presence of technical uncertainty, design needs to be revisited many times.  Often when using newer APIs the API is not documented or even worse, the code mechanism from the creator/vendor simply does not work and requires technical work arounds that materially affect the project schedule.

This will create a change request that will affect the project schedule.

An agile project builds requirements and design into every sprint.  This allows you to change directions when either requirements or technical decisions are uncertain.

This is the series of small waterfalls -- you are still doing Waterfall, but in an Agile way.

When you add up all the sprints of a project, you will realize that you have done the same project (i.e. the work is the work), but you have sliced it up differently.

So it is not a question of whether you will be doing Waterfall or not.  It is a matter of what is the sprint cycle that will support the requirements and technical uncertainty of your project.  If your project has little to no uncertainty, then your sprint cycle is the entire project.

If your project has requirements or technical uncertainty, then sprint cycles of 2-3 weeks are ideal.

^*Note: Similar to requirements and technical uncertainty are projects where there are significant dependencies on client interaction, so schedule uncertainty can also drive the need for agile project management.

---

Other articles in the "Loser" series

Want to see more sacred cows get tipped? Check out

• Comments are for Losers
• Defects are for Losers
• Debuggers are for Losers
• Efficiency is for Losers
• Testing departments are for Losers

Make no mistake, I am the biggest "Loser" of them all.  I believe that I have made every mistake in the book once :-)

Why Outsourcing Fails

Every year, we see corporations outsource operations only to pull back back some, if not all, of the operations. When this happens, the cost of pulling back operations or splitting things over multiple countries can leave you with significant operational issues and similar or worse costs.

Successful corporations understand the need to control costs in operations. The "spare no expense" philosophy of John Hammond in Jurassic Park is a one way ticket to disaster.

But, a focus on cost reduction is what leads corporations to face-fault when they outsource.

Corporations are seduced by the idea of potentially cutting costs dramatically; why pay expensive resources to do repetitive things when overseas people can do the same work for pennies?

The cardinal rule when outsourcing is:

Only operations that you completely understand and control can be outsourced 

To understand operations is to understand the costs of exceptions as well as how they are processed and escalated. Local resources deal efficiently with exceptions because they have informal relationships in the organization and use them to get things done efficiently; those relationships are not available to outsourced resources.

When operations gets outsourced, not only do foreign employees not understand how to deal with exceptions but also:

• They are often in another time zone 
• They don't have full access to local resources 
• They don't know who to contact inside the corporation 
• They don't understand local business rules 
• The network configuration can make it impossible to communicate properly 

Understanding an operational function means that you capture all details, including exceptions, in your information systems. Often, legacy information systems capture exception information as unstructured fields (i.e. memo fields) and off shore resources will not understand this.

99% of your operations might be straight forward, but the 1% of exceptions can at best lead to longer implementation times.  Best case scenario is that you will look bad to your customers and at worst, it will kill your bottom line.

If your information systems contain exception information in a structured way then at least you can eventually train foreign resources to resolve them. But unless exception information is captured so you can control it (i.e. not in general text fields) then it is the same thing as not having it.

When outsourced resources are under strict performance guidelines, they will do anything to clear their work queues. This often involves kicking back work they perceive as an exception as improperly specified; they will claim 'garbage in, garbage out'.

This will result in miscommunications and longer lead times to implement your services. Not only will delivery times, and errors go up, and your customers will get mad.

By the time you have fired the local employees, all the knowledge of the function has left the building and you are left with a broken system.

Key questions to get answers to:

• What does each exception in operations cost? 
• How often do they happen? 
• Can the information systems record exceptions in a way that you can track and control them? 
• Are the mechanisms that local resources use to resolve exceptions be used by outsourced resources? 

If you don't have a clear answer to each of these questions then outsourcing is likely to be a disaster. When you have a clear answer to the 3 questions and understand how you will address each exception then you will have much more success outsourcing.

Specifically, what is complexity?

People involved in software projects would say that software development is about understanding complexity.

What is complexity?

What can we do about it?

Complexity is easy to define at a high level, but people get vague when pressed for a precise definition.  Let's see if we can:

• quantify the problem
• define it
• suggest how to address it

Assume we need to develop a project of 50,000 lines of code ¹ , this project would take 5,000 man days to develop (scenario A) assuming 10 production lines of code per day ². Even if we are generous and assume 50 production lines of code per day per developer, the project would still take 1,000 man days to develop.

Now take that developed code and have a single developer re-enter it (scenario B). The developer is likely to be able to code at least 1,000 lines of code per day and it would only take about 50 man days to enter the entire program.

The difference between scenario A and scenario B is 950 days to create the same software system. So what are we doing in the other 950 days if it only takes 50 days to write the code?

95% of a project is about solving problems, not writing code

In scenario A we don't know which lines of code to write, time is expended in figuring out:

• determining what the actual requirements are
• translating the requirements into high level design
• designing an architecture sufficient to support the requirements
• translating the high level design into programming languages / libraries
• discovering where the code defects are
• rewriting code for incorrect functionality

While it might seem startling that 95% of our time is spend figuring out which lines of code to write, a little reflection will convince us that this is the case.  We mistakenly believe that projects fail because there is not enough time to write the code, but really, the problem is that we don't know what lines of code to write.

Projects do not fail because their is insufficient time to write code

When you see developers furiously coding at their desks, they are spending most of their time thinking.

Most developers are unable to sit and plan away from the computer, so when you see them at their keyboards they are primarily thinking. They will write and rewrite sections of code as they think through the problems (see Not Planning is for Losers).

Where does Complexity Come From?

Now that we have quantified how much time is spent solving complexity, where does it come from?

Complexity comes from team resources not understanding how to solve the problems they face. When confronted with a problem the team must think about how to solve it.

Each problem is like figuring out a path in a maze, it takes time to solve it initially.  Once you have solved the problem, it is easy to avoid all the dead ends and go straight to a solution.

We spend 95% of our time considering alternatives and hitting dead ends.

Complexity is not a issue if enough time has been budgeted in the project for the team to think things through. Complexity is a problem when you need to think through issues and insufficient time is budgeted.

Projects run late mainly because of two reasons:

1. Insufficient time
• You did not use formal estimation techniques and underestimate the project
• You use formal estimates and executives push for a delivery date independent of the problems to be solved in the project
2. Insufficient requirements
• You estimated sufficient time for your team's skill level to solve the problems but you did not have all the requirements

Insufficient time

It is pretty clear why insufficient time causes projects to fail, but here are some specific statistics.  Often executives reject formal estimates and then push for project deadlines independent of the work to be done (see Why Senior Management Declared Deadlines lead to Disaster)

Rejection of estimates for business reasons lead to productivity down 15% and quality down 21% 

Excessive schedule pressure leads to productivity down 16% and quality down 23% 

Insufficient Requirements

The second situation happens when you underestimate requirement complexity.  On a regular basis projects are launched where only a fraction of the requirements are gathered.  Once the project starts there is a mad dash to capture the missing requirements, i.e scope creep (see Shift Happens). 

Project plans are often based on the initial set of requirements and the project deadline is not adjusted when you discover that there are missing requirements.

Failure to estimate requirement changes to schedule lead to productivity down 16% and quality down 22% 

Why We Underestimate Requirement Complexity

Human beings are wired to underestimate complexity.  For example, suppose that a customer says that they want a car, we all know what a car is, correct?  Executives often talk in such generalities when they don't understand why the estimates are high.

Look at the following cars:

Clearly these cars are not the same, and the cost and time required to build them is not the same. Abstractly, we all understand what a car is and we can discuss this in general.  But when we start drilling into the details, there are many questions to be answered, such as:

• How fast does the car need to go?
• How many people does it need to seat?
• What kind of fuel economy do you want?
• What safety features are required?
• How sophisticated do the controls need to be?

In the rush to get projects started, we skimp on the requirements process and many questions are not answered.  But at some point the developers need that information so that they can solve their problems and the requirements need to be gathered.  Quite often projects run out of time simply because they never budgeted time to gather enough requirements.

Skill and Complexity

There is a clear trade off between the time required to solve a complex problem and the skill level of the team.  A problem is complex when you have never seen it before, but once you have solved it, then you always see the solution.

Just like the maze shown here, the first time might take a fair amount of time to figure out the solution.  If you have to solve the problem again then you will go directly to the solution without dead ends.

When gathering requirements, the better the analyst understands the domain, the less likely they are to miss requirements or have inconsistent requirements.  This eliminates the likelihood of inconsistent requirements and increases the probability that enough requirements are captured to start the project.  

When developers have experience with the domain then they take much less time to break the requirements into high level designs.  When they have experience with the language being used and the libraries they can translate that high level design into a low level design and code much quicker.


Solving Complexity

So successful software projects recognize that complexity amounts to the thinking time required for the team to figure out problems they have not solved before.

The more experienced a resource, the less time they need to think about the final solution.  The less experienced resources need more time to think through problems, especially when they hit dead ends.

• For the business analysts and product managers
• This is about understanding the subject matter and making sure to gather enough consistent requirements
• For the developers 
• This is about understanding the subject matter for high level design, and understanding the language and libraries for implementation
• For the project managers 
• This is about understanding how capable the team is and making sure that sufficient time is budgeted for them to solve all problems
• For the IT executives 
• This is about understanding that any given project team needs sufficient time to solve problems and not caving into executive pressure to arbitrarily shorten projects

References

• Jones, Capers.SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.

---

¹ Lines of code is a terrible metric and is only used here to illustrate a point about complexity. Software measurement should use function points or another relevant metric.

² Brooks, Frederick Phillips. “The Mythic Man Month”, 1995. ISBN 0-201-00650-2

Do Project Managers need Domain Experience?

Opinions vary on whether a project manager needs to have domain experience.  Certainly project managers that do not have domain experience will be the first to say that domain experience is not necessary as long as they have access to excellent subject matter experts.

I would advocate a more nuanced position; that is, a project manager does not need domain experience IF his subject matter experts understand the risks and dependencies that are inherent to the domain.

Let's go through a couple of personal projects that I have been involved with where the project manager did not have domain experience.

Telco Project

I am currently involved in a project that involves a LAN/WAN/WIFI upgrade of a large customer for a large telecommunications company.  The project manager does not have domain expertise in networks and is counting on the subject matter experts to provide him sufficient input to execute the project.

The subject matter experts are so advanced in their knowledge of networks that they no longer understand what beginners (i.e. the project manager) do not know.  They have assumed that when they indicate things to the project manager that he understands what they mean and will take appropriate actions.

The project manager is continually running into situations where he did not understand the implications of certain risks and dependencies.  This has caused a certain amount of rework and delays.

Fortunately, this is not a project with tremendous amounts of risk or dependencies so the project will be late but will succeed.

Mobile Handset Project

In the distant past, I was part of a team that was building a mobile POS terminal that worked over cellular (GSM, CDMA).  The project manager in this situation did not have domain experience and was counting on the subject matter experts.  In this case, the subject matter experts were very good at general design, but not experts in building cellular devices.

Because the subject matter experts were not specialists, they knew most of the key principles of designing mobile handsets but did not understand all the nuances of handset design.  There were several key issues required by practical handset manufacturing that were overlooked by the generalists and ended up creating such a strong cost over-run that the start-up went out of business.

Sumary

In the first project, the subject matter experts were extremely good, however, the project manager failed to understand the implication of some of their statements and this introduced large delays in the project.

In the second project, the subject matter experts were generalists and did not understand all the risks and dependencies of the project.  The project manager (and start-up) were doomed to fail because "you don't know what you don't know". Both these projects show that a project can be delayed or fail because a project manager does not have domain experience.

Conclusion

So if a project does not have many uncertainties and dependencies then it is extremely likely that the project manager does not require domain experience and can rely to some degree on his subject matter experts.

However, if the project has complex uncertainties and/or dependencies then a good project manager without domain experience is likely to find himself in a position where the consequences of not understanding the uncertainties and dependencies will either introduce serious rework or torpedo the project.

---

See also

• Schedule Risk is a Red Herring!!!
• Uncertainty trumps Risk in Software Development

Infeasible Projects: Executive Ignorance or IT Impotence?

Infeasible software projects are launched all the time and teams are continually caught up in them, but what is the real source of the problem?

There are 2 year actual projects for which the executives set a 6 month deadline.

The project is guaranteed to fail but is this due to executive ignorance or IT impotence?

There is no schedule risk in an infeasible project because the deadline will be missed.  Schedule risk only exists in the presence of uncertainty (see Schedule Risk is a Red Herring!!!)

As you might expect, all executives and IT manager share responsibility for infeasible projects that turn into death marches.  Learn about the nasty side effects Death March Calculus. The primary causes for infeasible projects are:

• Rejection of formal estimates
• No estimation or improper estimation methods are used

Rejecting Formal Estimates

This situation occurs frequently; an example would be the Denver Baggage Handling System (see Case Study).

The project was automatically estimated (correctly) to take 2 years; however, executives declared that IT would only have 1 year to deliver.

Of course, they failed¹.

The deadline was rejected by executives because it did not fit their desires.  They could not have enjoyed the subsequent software disaster and bad press.

When executives ignore formal estimates they get what they deserve.  Formal estimates are ignored because executives believe through sheer force of will that they can set deadlines.

If IT managed to get the organization to pay for formal tools for estimating then it is not their problem that the executives refuse to go along with it.

Improper Estimation Methods

The next situation that occurs frequently is using estimation processes that have low validity.  Estimation has been extensively studied and documented by Tom DeMarco, Capers Jones, Ed Yourdon, and others.

Improper estimation methods will underestimate a software project every time. Fast estimates will be based on what you can think of, unfortunately, software is not tangible and so what you are aware of is like the tip of an iceberg.

None of this prevents executives demanding fast estimates from development.  Even worse, development managers will cave in to ridiculous demands and actually give fast estimates.

Poor estimates are guaranteed to lead to infeasible projects (see Who needs Formal Measurement?) Poor estimates are delivered by IT managers that:

• Can't convince executives to use formal tools
• Give in to extreme pressure for fast estimates

Infeasible projects that result from poor estimates are a matter of IT impotence.

Conclusion

Both executive ignorance and IT impotence lead to infeasible projects on a regular basis because of poor estimates and rejecting estimates; so there is no surprise here.

However, infeasible projects are a failure of executives and IT equally because we are all on the same team. It is not possible for part of the organization to succeed if the other one fails.

Possibly a greater share of problem is with IT management.  After all, whose responsibility is a bad decision -- the guys that know what the issues are or the ones that don't.

If a child wants ice cream before they eat dinner then whose fault is it if you cave in and give them the ice cream?

Unfortunately, even after 60 years of developing software projects, IT managers are either as ignorant as the executives or simply have no intestinal fortitude.

Even when IT managers convince executives of the importance of estimating tools, the estimates are routinely discarded because they do not meet executive expectations.

Rejection of automated estimates: productivity -16%, quality -22%

Until we can get a generation of IT managers that are prepared to educate executives on the necessity of proper estimation and be stubborn about holding to those estimates, we are likely to continue to have an estimated $3 trillion in failures of software projects every year.

---

End Notes

¹For inquiring minds, good automated estimation systems have been shown to be within 5% of time and cost on a regular basis. Contact me for additional information.

References

• Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.
• Jones, Capers. The Economics of Software Quality. 2011
• Kahneman, Daniel. Thinking, Fast and Slow. 2011

Failed? You get what you deserve!

Consider this, few projects fail because of unusual or unforeseen problems.  If you are trying to go so fast that you don't do your due diligence or decide to skip steps then -- "You get what you deserve!".

Projects succeed because they do not rely on force of personality and luck.

Only 3 of 10 software projects succeed (see Understanding Your Chances). Success happens when you implement best practices and avoid worst practices; not just talk about them and convince yourself that you are doing them.

Therefore 7 out of 10 projects fail, even if you redefine 'challenged projects' as political victories; these projects are often severely over time and budget.

It is estimated that between $3 trillion and $6 trillion dollars are wasted every year in IT; by organizations that "don't know, and don't know that they don't know", i.e. what you don't know can definitely hurt you.

What is amazing is that failures do not prompt the incompetent to learn why they failed.  After the post-fail finger-pointing ceremony, people just dust themselves off and rinse and repeat.

There is never enough time to do the project correctly, but there is always enough time to do it again.

Ingredients of a Successful Project

Successful software projects have the following characteristics:

• Effective capital budgeting
• Proper project sizing
• Estimate uncertainty
• A focus on defect removal

Capital Budgeting

Capital budgeting means having business cases for major projects and allocating capital to the most promising projects; a little due diligence prevents bad projects from starting.

For example, RJ Reynolds invested $325 million to develop smokeless cigarettes when they knew in the initial pilot that the cigarette tasted bad and no one would buy them.

Project Sizing

If a project has a good business case, then the next step is to figure out how big the project is.  This requires that you expand  the requirements sufficiently to estimate time and cost.

When it comes to software -- size matters.

Not only does size matter, you need to understand the level of uncertainty (requirements, technical, or skill) tied to your project to understand how much re-work or discovery will be necessary.

• Re-engineering a product with new technologies?  (high technical uncertainty)
• Building a new product with existing technologies? (high requirements uncertainty)
• Building software with a team unfamiliar with the technology? (have high skills uncertainty)

Skipping the time to get estimates is a major cause of project failure

There are still executives that give engineering a few hours or a few days to estimate major projects and wonder why the projects go late or get cancelled.

Estimate Uncertainty

Project methodology needs to take into account expected uncertainty.

Projects with low uncertainty can be handled with a traditional project methodology; these projects can have their work breakdown structures elaborated and dependencies can be worked out ahead of time.

Projects with moderate to high uncertainty will require rework and discovery.  This must be built into the project plan or you need to switch to an Agile methodology.  If you don't then there will be missing tasks on the project plan as well as task duration that is under-estimated.

Project tasks get stuck at 90% complete because the sources of uncertainty are not established a priori

Executing a project with personnel that are not competent with the technologies that you want to use is essentially professional malpractice.  If you must use existing personnel that don't know the technologies you want then you need to build in a large margin for training and rework over the project; these projects will take at least 2x as long as your worst estimate.

Focus on Early Defect Removal

Developer's say: "There is never enough time to write the code".  Statistics say, "There is never enough time to find all the defects" (see The Programmer Productivity Paradox).  To be successful you need to find defects as quickly as possible, especially since:

• If it costs X to find a defect pre-test 
• It costs 10X to find it in QA
• It costs 100X to find it once deployed in the field

You do the math, it costs less to find defects before they get to QA.  You are kidding yourself if you think that it is cost effective to have huge QA departments (see The Cost of Not Doing Quality).

Conclusion

Building software reliably and repeatably is not difficult.  What is difficult is getting organizations to realize that there is a minimum set of processes that must be followed to have a successful project.

Organizations regularly skip one or more steps and wonder projects fail to come together and succeed.

Are Debuggers Crutches?

Debuggers have become powerful tools, but like a drug have we become too dependent on them? Since poor developers spend 25 times more time in the debugger there is a likelihood that people are zoning out in the debugger instead of using other methods.

Defects are common, but they are not not necessary.  They find their way into code because:

• Code pathways are not planned
• Developers are inattentive when coding
• Developers do not understand the requirements

Defect correction is only possible if you understand the code pathways, debuggers are not the best way to do this

Debuggers are commonly used by developer's to understand a problem, but just because they are common does not make them the best way to find defects.  I'm not advocating a return to "the good old days" but there was a time when we did not have debuggers and we managed to debug programs.

Note: in embedded systems it is very hard to get feedback if you do not use a debugger (or hardware debugger).  This article is not addressed to embedded developers who rarely have an alternative to a debugger.

Avoid Defects

The absolute best way to remove defects is simply not to create them in the first place. You can be skeptical, but things like  the Personal Software Process (PSP) have been used practically to prevent 1 of every 2 defects from getting into your code.  Over thousands of projects:

The Personal Software Process increases productivity by 21% and increases code quality by 31%¹

A study conducted by NIST in 2002 reports that software bugs cost the U.S. economy $59.5 billion annually. This huge waste could be cut in half if all developers focused on not creating defects in the first place.

Not only does the PSP focus on code planning, it also makes developers aware of how many defects they actually create.  Here are two graphs that show the same group of developers and their defect injection rates before and after PSP training.2

Before PSP training	After PSP training

Finding Defects

Using a debugger to understand the source of a defect is definitely one way.  But if it is the best way then why do poor developers spend 25 times more time in the debugger than a a good developer? (see No Experience Required!)

Poor developers spend a week in the debugger for every 2 hours that good developer does

No one is saying that debuggers do not have their uses.  However, a debugger is a tool and is only as good as the person using it.  Focus on tools obscures lack of skill (see Agile Tools do NOT make you Agile)

If you are only using a debugger to understand defects then you will be able to remove a maximum of about 85% of all defects, i.e. 1 in 7 defects will always be present in your code.

Would it surprise you to learn that their are organizations that achieve 97% defect removal³?  Software inspections take the approach of looking for all defects in code and getting rid of them. Learn more about software inspections and why they work here:

• Inspections are not Optional
• Software Professionals do Inspections

Software inspections increase productivity by 21% and increases code quality by 31% 

Even better, people trained in software inspections tend to inject fewer defects into code. When you become adept at parsing code for defects then you become much more aware of how defects get into code in the first place.

But interestingly enough, not only will developers inject fewer defects into code and achieve defect removal rates of up to 97%, in addition:

Every hour spent in code inspections reduces formal QA by 4 hours

Conclusion

As stated above, there are times where a skilled professional will use a debugger correctly.  However, if you are truly interested in being a software professional then:

• You will learn how to plan and think through code before using the keyboard
• You will learn and execute software inspections
• You will learn techniques like PSP which lead to you injecting fewer defects into the code

You are using a debugger as a crutch if it is your primary tool to reduce and remove defects

Related Articles

Want to see more sacred cows get tipped? Check out:

Comments are for Losers Efficiency is for Losers Debuggers are for Losers Testing departments are for Losers
	Moo?

Make no mistake, I am the biggest "Loser" of them all.  I believe that I have made every mistake in the book at least once :-)

References

• Gilb, Tom and Graham, Dorothy. Software Inspections
• ¹Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.
• ³Jones, Capers. The Economics of Software Quality. 2011
• Radice, Ronald A. High Quality
• ²Watts, Humphrey. Introduction to the Personal Software Process

No Business Case == Project Failure

A business case comes between a bright idea for a software project and the creation of the software project. 

• To - idea to have a project is born
• Tcheck - formal or informal business case
• Tstart - project is initiated
• Tend - project finishes successfully or is abandoned

Not all ideas for software projects make sense.  In the yellow zone above, between idea and project being initiated, some due diligence on the project idea should occur.  This is where you do the business case, even if only informally on the back of a napkin.

The business case is where you pause and and estimate  whether the project is worth it, i.e. will this project leave you better off than if you did not do it. For those who want precise definitions the project should be NPV +ve.  In layman's terms, the project should leave the organization better off on it's bottom line or at least improve skill levels so that other projects are better off.

Projects that do not improve skills or the bottom line are a failure.

Out of 10 software projects (see Understanding your chances):

• 3 are successful
• 4 are challenged, i.e. over cost, over budget, or deliver much less functionality
• 3 will fail, i.e. abandoned

This means that the base rate of success for any software project is only 3 out of 10. Yet executives routinely execute projects assuming that they can not fail, even though the project team may know that the project will be a failure from day 1.

Business cases give executives a chance to stop dubious projects before they start. (see Stupid is as Stupid Does) Understanding how formal the business case needs to be comes down to uncertainty. There are three key uncertainties with every project:

• Requirements uncertainty
• Technical uncertainty
• Skills uncertainty

When there is a moderate amount of uncertainty in any of these three areas then a formal business case with cash flows and risks needs to be prepared.

Requirements Uncertainty

Requirements uncertainty is what leads to scope shift (scope creep).

The probability of a project failing is proportional to the number of unknown requirements when the project starts (see Shift Happens).

Requirements uncertainty is only low for two particular projects:

1. re-engineering a project where the requirements do not change
2. the next minor version of a software project. 

For all other software projects the requirements uncertainty is moderate and a formal business case should be prepared.

Projects new to you have high requirements uncertainty.

Technical Uncertainty

Technical uncertainty exists when it is not clear that all requirements can be implemented using the selected technologies at the level of performance required for the project. Technical uncertainty is only low when you have a strong understanding of the requirements and the implementation technology.  Uncertainty and risk are two different animals (see  Uncertainty Trumps Risk in Software Development)

When there is only a moderate understanding of the requirements or the implementation technology then you will encounter the following problems:

• Requirements that get clarified late in the project that the implementation technology will not support
• Requirements that can not be implemented once you improve your understanding of the implementation technology

Therefore technical uncertainty is high when you are doing a project for the first time and requirement uncertainty is high.  Technical uncertainty is high when you are using new technologies, i.e. shifting from Java to .NET or changing GUI technology.

Projects with new technologies have moderate to high uncertainty.

Skills Uncertainty

Skills uncertainty comes from using resources that are unfamiliar with the requirements or the implementation technology.  Skills uncertainty is a knowledge problem.

Skills uncertainty is only low when the resources understand the current requirements and implementation technology.

Resources unfamiliar with the requirements will often implement requirements in a suboptimal way when requirements are not well written.  This will involve rework; the worse the requirements are understood the more rework will be necessary.

Resources unfamiliar with the implementation technology will make mistakes choosing implementation methods due to lack of familiarity with the philosophies of the implementation libraries.  Often after a project is complete, resources will understood that different implementation tactics should have been used.

Formal or Informal Business Cases?

An informal business case is possible only if the requirements, technical, and skills uncertainty is low.  This only happens in a few situations:

• Replacing a system where the requirements will be the same and the implementation technology is well understood by the team
• The next minor version of a software system

Every other project requires a formal business case that will quantify what kind of uncertainty and what degree of uncertainty exists in the project.  At a minimum project managers facing moderate to high uncertainty should be motivated to push for a business case (see Stupid is as Stupid Does).

Here is a list of projects that tend to be accepted without any kind of real business case that quantifies the uncertainties:

• Change of implementation technology
  • Moving to object-oriented technology if you don't use it
  • Moving from .NET to Java or vice versa
• Software projects by non-software companies
• Using generalists to implement technical solutions
• Replacing systems with resources unfamiliar with the requirements
  • Often happens with outsourcing

Projects with moderate to high risks and no business case are doomed to fail.

Related articles

• No Experience Necessary!
  • Surprisingly there is no correlation between years of experience and productivity.  This has been verified over 40 years.
• Stop it! No, Really Stop It!
  • 5 worst practices that software organizations commonly practice that they need to stop right away

It's not a bug, it's...

When does a bug become a bug?

Who decides that it is a bug?

How many legs does a lamb have if I say the tail is a leg?  The answer is 4, just because I say the tail is a leg does not make it a leg!

Bugs should be obvious, but we say It's not a bug, it's a feature because often it isn't obvious.  Watson Humphrey felt that we should use the term defect and not bug because most people don't take bugs seriously, so let's use the term defect instead.

So when does a defect become a defect?

• When quality assurance tells you that you have a defect?
• When product management says that it is a defect?
• When the customer says that it is a defect?

The answer is: none of the above. Now it might turn out that there is a problem and that code needs to change, but a defect only exists if:

code behaves differently than the requirements specification

This is important because most systems are under specified (if they are specified at all :-) ). Code can only be considered a defect if it differs from the specification.  We call defects undocumented features because we know that the problem is that the requirements were never written.

Incomplete and Inconsistent Requirements

Many organizations do not create sufficiently complete requirements before starting development, either because they don't know how to capture requirements properly or because they don't have resources capable of capturing complete requirements.

Incomplete (and inconsistent) requirements and unrealistic deadlines often force developers into making decisions about how to implement features.  The end result is that developers are regularly told that they have defects in their code.

While this process is common, it is destructive.  When requirements are under specified and inconsistent developers end up needing to perform serious rework. The rework will can require dramatic changes that will impact the architecture of the code.

The time required to find a work around (if it is possible) is rarely included in the project plan. Complicating matters is that the organizations that are reluctant to spend time creating requirements also tend to underestimate their projects.  This puts tremendous pressure on the engineering department to deliver; this promotes the 5 worst practices in software development (see Stop It! No… really stop it.)

When poorly or undocumented systems require changes that are not specified we should  call them change requests rather than defects.

Only 54% of Issues are Resolved by Engineers

The attitude that all defects must be resolved by the engineering department is severely misguided.  Analysis by Capers Jones of over 18,000+ projects shows that only about 54% of all defects can be resolved by the engineers! (only the 3 highlighted rows below)

Defect Role Category	Frequency	Role
Requirements defect	9.58%	BA/Product Management
Architecture or design defect	14.58%	Architect
Code defect	16.67%	Developer
Testing defect	15.42%	Quality Assurance
Documentation defect	6.25%	Technical Writer
Database defect	22.92%	Data base administrator
Website defect	14.58%	Operations/Webmaster

This means that precious time will be wasted assigning issues to developers that they can not resolve.  The time necessary to redirect the issue to the correct person is a major contributing factor to fire-fighting

Getting Control of the Defect Process

For most organizations fixing the defect process involves understanding and categorizing defects correctly. Organizations that are not tracking the different sources of defects probably have a bug tracker that has gone to hell.  Here is how you can fix that problem, see Bug Tracker Hell and How To Get Out!

At a minimum you need to implement the requirements defect, once you identify issues that are caused by poor requirements it will shine the white hot light of shame onto the resources that are capturing your requirements.

Once you realize how many requirements defects exist in your system you can begin to inform senior management about the requirements problem.

Reducing Fire-Fighting

The best way to reduce fire fighting is to start writing better requirements (or writing requirements :-) ).  To do so you need to figure out which of the following are broken:

1. Not enough time is allocated to the requirements phase
2. Unskilled people are capturing your requirements

In all likelihood both of these issues need to be fixed in your organization.  When requirements are incomplete and inconsistent you will have endless fire-fighting meetings involving everyone (see Root cause of 'Fire-Fighting' in Software Projects)

Stand your ground if someone tells you that you have coded a defect when there is no documentation for the requirement.