Friday 27 June 2014

Failed? You get what you deserve!

Consider this, few projects fail because of unusual or unforeseen problems.  If you are trying to go so fast that you don't do your due diligence or decide to skip steps then -- "You get what you deserve!".

Projects succeed because they do not rely on force of personality and luck.

Only 3 of 10 software projects succeed (see Understanding Your Chances). Success happens when you implement best practices and avoid worst practices; not just talk about them and convince yourself that you are doing them.

Therefore 7 out of 10 projects fail, even if you redefine 'challenged projects' as political victories; these projects are often severely over time and budget.

It is estimated that between $3 trillion and $6 trillion dollars are wasted every year in IT; by organizations that "don't know, and don't know that they don't know", i.e. what you don't know can definitely hurt you.

What is amazing is that failures do not prompt the incompetent to learn why they failed.  After the post-fail finger-pointing ceremony, people just dust themselves off and rinse and repeat.

There is never enough time to do the project correctly, but there is always enough time to do it again.


Ingredients of a Successful Project


Successful software projects have the following characteristics:
  • Effective capital budgeting
  • Proper project sizing
  • Estimate uncertainty
  • A focus on defect removal

Capital Budgeting

Capital budgeting means having business cases for major projects and allocating capital to the most promising projects; a little due diligence prevents bad projects from starting.

For example, RJ Reynolds invested $325 million to develop smokeless cigarettes when they knew in the initial pilot that the cigarette tasted bad and no one would buy them.

Project Sizing

If a project has a good business case, then the next step is to figure out how big the project is.  This requires that you expand  the requirements sufficiently to estimate time and cost.

When it comes to software -- size matters.

Not only does size matter, you need to understand the level of uncertainty (requirements, technical, or skill) tied to your project to understand how much re-work or discovery will be necessary.

  • Re-engineering a product with new technologies?  (high technical uncertainty)
  • Building a new product with existing technologies? (high requirements uncertainty)
  • Building software with a team unfamiliar with the technology? (have high skills uncertainty)

Skipping the time to get estimates is a major cause of project failure

There are still executives that give engineering a few hours or a few days to estimate major projects and wonder why the projects go late or get cancelled.

Estimate Uncertainty

Project methodology needs to take into account expected uncertainty.

Projects with low uncertainty can be handled with a traditional project methodology; these projects can have their work breakdown structures elaborated and dependencies can be worked out ahead of time.

Projects with moderate to high uncertainty will require rework and discovery.  This must be built into the project plan or you need to switch to an Agile methodology.  If you don't then there will be missing tasks on the project plan as well as task duration that is under-estimated.

Project tasks get stuck at 90% complete because the sources of uncertainty are not established a priori

Executing a project with personnel that are not competent with the technologies that you want to use is essentially professional malpractice.  If you must use existing personnel that don't know the technologies you want then you need to build in a large margin for training and rework over the project; these projects will take at least 2x as long as your worst estimate.

Focus on Early Defect Removal

Developer's say: "There is never enough time to write the code".  Statistics say, "There is never enough time to find all the defects" (see The Programmer Productivity Paradox).  To be successful you need to find defects as quickly as possible, especially since:

  • If it costs X to find a defect pre-test 
  • It costs 10X to find it in QA
  • It costs 100X to find it once deployed in the field

You do the math, it costs less to find defects before they get to QA.  You are kidding yourself if you think that it is cost effective to have huge QA departments (see The Cost of Not Doing Quality).

Conclusion


Building software reliably and repeatably is not difficult.  What is difficult is getting organizations to realize that there is a minimum set of processes that must be followed to have a successful project.

Organizations regularly skip one or more steps and wonder projects fail to come together and succeed.

Monday 9 June 2014

Don't be a Slave to Your Tools

Developers attach quickly to tools because they are concrete and have well defined behavior.  It is easier to learn a tool than to learn best practices or methodology. Tools only assist in solving problems, they can't solve the problem by themselves.

A developer who understands the problem can use tools to increase productivity and quality. Poor developers don't invest the time or effort to understand how to code properly and avoid defects.  They spend their time learning how to use tools without understanding the purpose of the tool or how to use it effectively.

To some degree, this is partially the fault of the tool vendors.  The tool vendors perceive an opportunity to make $$$$$ based on providing support for a common problems, such as:
  • defect trackers to help you manage defect tracking
  • version control systems to manage source code changes
  • tools to support Agile development (Version One, JIRA)
  • debuggers to help you find defects
There are many tools out there, but let's just go through this list and point out where developers and organizations get challenged.  Note, all statistics below are derived from over 15,000 projects over 40 years.1

Defect Trackers

Believe it or not, some organizations still don't have defect tracking software. I've run into a couple of these companies and you would not believe why. The effect of not having a defect tracking system is pretty severe, and there is evidence to prove this.

Inadequate defect tracking methods: productivity -15%, quality -21%

So we are pretty much all in agreement that we need to have defect tracking; we all know that the ability to manage more than a handful of defects is impossible without some kind of system

Automated defect tracking tools: productivity +18%, quality +26%

The problem is that developers disagree over what might be the best defect tracking system. The real problem is that almost every defect tracking system is poorly set-up, leading to poor results. Virtually every defect tracking system when configured properly will yield tremendous benefits. The most common pitfalls are:
  • Introducing irrelevant attributes into the defect lifecycle status, i.e. creation of statuses like deferred, won't fix, or functions as designed
  • Not being able to figure out if something is fixed or not
  • Not understanding who is responsible for addressing a defect
The tool vendors are happy to continue to provide new versions of defect trackers. However, using a defect tracker effectively has more to do with how the tool is used rather than which tool is selected.

One of the most fundamental issues that organizations wrestle with is what is a defect?  A defect only exists if the code does not behave according to specifications. But what if there are no specifications or the specifications are bad?  See It's not a bug, it's... for more information.

Smart organizations understand that the way in which the defect tracker is used will make the biggest difference.  Discover how to get more out of you defect tracking system in Bug Tracker Hell and How to Get Out.

Another common problem is that organizations try to manage enhancements and requirements in the defect tracking system.  After all whether it is a requirement or a defect it will lead to a code change, so why not put all the information into the defect tracker?  Learn why managing requirements and enhancements in the defect tracking system is foolish in Don't manage enhancements in the bug tracker.

Version Control Systems

Like defect tracking systems most developers have learned that version control is a necessary hygiene procedure.  If you don't have one then you are likely to catch a pretty serious disease (and at the least convenient time)

Inadequate change control: productivity -11%, quality -16%

Virtually all developers dislike version control systems and are quite vocal about what they can't do with their version control system.  If you are the unfortunate person who decided on the version control system,  just understand that there are hordes of developers out their cursing you behind your back.

Version control is simply chapter 1 of the story.  Understanding how to chunk code effectively, integrate with continuous build technology, and making sure that the defects in the defect tracker refers to the correct version are just as important as the choice of version control system.

Tools to support Agile

Sorry Version One and JIRA, the simple truth is that using an Agile tool does not make you agile, see this.

These tools are most effective when you actually understand Agile development.  One of my most effective Agile implementations only used Google docs in the implementation.

Enough said.

Debuggers

I have written extensively about why debuggers are not the best primary tools to track down defects. So I'll try a different approach here. :-)

One of the most enduring sets of ratios in software engineering has been 1:10:100.  That is, if the cost of tracking down a defect pre-test (i.e. before QA) is 1, then it will cost 10x if the defect is found by QA, and 100x if the defect is discovered in deployment by your customers. Most debuggers are invoked when the cost function is in the 10x or 100x part of the process.

It is not that I dislike debuggers -- I simply believe in using pre-test defect removal strategies because they cost less and lead to higher code quality. Pre-test defect removal strategies include:
  • Planning code, i.e. PSP
  • Test driven development, TDD
  • Design by Contract (DbC)
  • Code inspections
  • Pair programming for complex sections of code
You can find more information about this in:

Seldom Used Tools

Tools that can make a big difference but many developers don't use them:

Automated static analysis: productivity +21%, quality +31%

Automated sizing in function points: productivity +17%, quality +24%

Automated quality and risk prediction: productivity +16%, quality +23%

Automated test coverage analysis: productivity +15%, quality +21%

Automated deployment support: productivity +15%, quality +20%

Automated cyclomatic complexity computation: productivity +15%, quality +20%

Important Techniques with No Tools

There are a number of techniques available in software development that tool vendors have not found a way to monetize on. These techniques tend to be overlooked by most developers, even though they can make a huge difference in productivity and quality.

The Personal Software Process (PSP) and Team Software Process (TSP) were developed by Watts Humphrey, one of the pioneers of building quality software.

Personal software process: productivity +21%, quality +31%2

Team software process: productivity +21%, quality +31%3
Code inspections: productivity +21%, quality +31%4

Requirement inspections: productivity +18%, quality +27%4

Formal test plans: productivity +17%, quality +24%

Function point analysis (IFPUG): productivity +16%, quality +22%

The importance of inspections is covered in:

Conclusion

There is definitely a large set of developers that assume that using a tool makes them competent.

 The reality is that learning a tool without learning the principles that underlie the problem you are solving is like assuming you can beat the great Michael Jordan at basketball just because you have great running shoes.

Learning tools is not a substitute for learning how do do something competently.

Competent developers are continually learning about techniques that lead to higher productivity and quality, whether or not that technique is supported by a tool.

References