Wednesday, 5 June 2013

Let the Machines Take Over

Whether you believe that the machines will take over or not will always be up for debate (until they actually take over :-). What is not up for debate is that many processes for software creation have been automated and can help improve productivity and quality in every phase of a software project.

Here are some statistics about how useful automation can be and give you some ammunition when you confront management about the need for these tools.

These results have been generated by Capers Jones who has been collecting data for over 15,000 projects for about 40 years.  Full results shown in the paper in the references.

Requirements Analysis

There have been tools available for tracing requirements for quite some time.  Generally the only companies forced to provide requirements traceability, but this makes sense to do on all large projects:

Automated requirements tracing can raise productivity by 12.89% and quality by 17.8%

There are tools that can help size programs in terms of function points, which then allows you to get relatively accurate estimates of a project`s cost and time before the project is executed.

Automated sizing in function points can raise productivity by 16.5% and quality by 23.7%

Development

There are several tools that are useful during development, but by far the biggest bang for your buck comes in the form of automated static analysis.

Automated static analysis can raise productivity by 20.9% and quality by 30.9%

There are now static analysis tools available for virtually all current languages that are cost effective and can help to locate those hard to find occasional defects.  You can find a list here of automated tools for C/C++, Java, JavaScript, Objective-C, Perl, PHP, and Python.

Then getting cyclomatic complexity counts for your code can be done in an automated fashion.  Studies have shown that files with a count of 74+ were 98% likely to have defects.

Automated cyclomatic complexity analysis can raise productivity by 14.5% and quality by 19.5%

Performance is a tricky thing, we will talk about it often but rarely do we actually do something about it. Automated performance analysis allows you to determine if there will be production problems before deploying new code.

Automated performance analysis can raise productivity by 12.5% and quality by 19.5%

There are tools for restructuring code automatically, many of these are built directly into today's IDEs.  There are many stand alone tools for automated restructuring, some are shown here.

Automated restructuring can raise productivity by 8.0% and quality by 11.7%

Automated unit testing is the easiest way to detect defects early.  If your developers are using Test Driven Development (TDD) and any continuous integration tool  (i.e, Hudson or Jenkins) then you can find out with each build if a defect has crept into the code.

Automated unit testing can raise productivity by 16.5% and quality by 23.8%

Testing

Believe it or not I was working with a team that was not using version control or defect tracking last year. So there are clearly still some organizations not using defect tracking.

Automated defect tracking can raise productivity by 17.5% and quality by 25.9%

Testing the percentage of the application pathways that are actually being tested is critical.  Most of your defects will lie in the pathways that you can not test.

Automated test coverage analysis can raise productivity by 15.5% and quality by 21.2%

Test cases can be generated automatically in some situations.

Automated test case generation can raise productivity by 15.0% and quality by 20.0%

Deployment

The next two tools are not about technology that you acquire, rather it is about building tools to support critical deployment functions.  When the pressure is on it is tough to think about creating applications to manage configuration in an automated way.  When management tells you that it would be a luxury to build these tools then show them these results.

Some organizations rely on manually updating configurations, but can be error prone if there are many values to update.

Automated configuration control can raise productivity by 16.4% and quality by 23.5%

Also, when automated configuration tools don`t exist then you often don`t have automated deployment tools.

Automated deployment support can raise productivity by 14.6% and quality by 19.6%

Conclusion

One of the most cost effective ways to improve productivity and quality in your software development is to automate any of the issues above.  In most cases you can find a vendor that will provide you support; for deployment you can make the case that building automated tools is the way to go.

References
1N.B. All productivity and quality percentages were derived over 15,000+ actual projects

Articles in the "Loser" series
Moo?

Want to see sacred cows get tipped? Check out:
Make no mistake, I am the biggest "Loser" of them all.  I believe that I have made every mistake in the book at least once :-)
Posted by at 1 comment:
Labels: , , , , , ,

Monday, 3 June 2013

To be, or not to be... Formal

What the heck is a formal process?  Is it in the category of you know it when you see it?

A formal process is a methodical way of tackling any repetitive process so that it can be handled in a standardized way.  The idea of formality is to reduce variation in output and minimize the chance of forgetting something.  Formal processes include the ideas of studying, planning, measurement, and process refinement; they cost more and take longer than just just jumping in and getting things done.

Imagine if they tried to make cars in an informal way

Informal, or not formal processes, are faster and cheaper because you get rid of the overhead of a formal process.  It makes sense to be informal when the variation of the process output is not very sensitive to studying, planning, measurement, or refinement.

We are all familiar with trying to use formality when it makes no sense, you simply end up with process for processes sake and waste time and money.

In software, formality has become synonymous with documentation, and frankly we all hate documentation. We don't like to produce it and we don't like to read it.  I've instructed hundreds of engineers and only a few people bother to read the requirements brick.  The way education is going right now I'm not even sure that university graduates can read anyways, but I digress... :-)

For example, documentation for requirements, design, and testing is the end result of a formal process, it is not the formal process itself.  It is smart not to produce more documentation than you need, but that does not mean that the formal processes that creates the documentation are optional.

Formal processes in software development cluster around two very simple ideas:
  • getting the correct requirements
  • elimination of defects
The problem is that formal processes only work when you do enough of them to make a difference.  That is not doing enough of a formal process is just as bad as doing too much.  For example, UML class diagrams will not only help you to plan code but also educate new developers on the team.

Producing UML diagrams is a formal practice; however, if you insist on producing UML diagrams for everything then you will end up spending a significant amount of time creating the diagrams to get a diminishing benefit.  Not having any UML diagrams will lead to code being developed more than once simply because no one has a good overview of the system.
Selective UML diagrams can really accelerated your development

Visualizing Formality

As you increase any formal practice the effect of that practice will increase, for example the more formal you are with UML diagrams the less defects you are likely to have:

However, as you increase formality, the costs of keeping track of all your artifacts goes up:
When you put the two graphs together you get this effect:
This shows that there is a cost to not having enough formality, i.e. no UML diagrams means that more code will get developed and less reuse means more defects.  However, producing too many UML diagrams will also have the cost of producing the UML diagrams and keeping them updated.  The reality is that for any formal practice there is a sweet spot where the minimal formality gives the lowest cost.

What that means is that productivity will be maximized in the sweet spot.

Remember that most formal practices are hygiene practices, these are things that no one wants to do but are really necessary to be highly productive and produce high quality code (see here for more on hygiene practices)

Statistics Behind Formal / Informal Practices

Formal practices can be applied to many things, but Capers Jones has measured the following formal practices (when properly executed) can have a positive effect on a project (15,000+ projects): 

Formal risk management can raise productivity by 17.8% and quality by 26.4%
Formal measurement programs can raise productivity by 20.0% and quality by 30.0%
Formal test plans can raise productivity by 16.6% and quality by 24.1%
Formal requirements analysis can raise productivity by 16.3% and quality by 23.2%
Formal SQA teams can raise productivity by 15.2% and quality by 20.5%
Formal scope management can raise productivity by 13.5% and quality by 18.5%
Formal project office can raise productivity by 11.8% and quality by 16.8%

A special category of formal practices are inspections:
Formal code inspections can raise productivity by 20.8% and quality by 30.9%
Formal requirements inspections can raise productivity by 18.2% and quality by 27.0% Formal design inspections can raise productivity by 16.9% and quality by 24.7%
Formal inspection of test materials can raise productivity by 15.1% and quality by 20.2%

More information on inspections can be found here:
Not being formal enough has its costs, and it is rarely neutral.  Here are some of the costs of not being formal enough:
Informal progress tracking can lower productivity by 0.5% and quality by 1.0%
Informal requirements gathering can lower  productivity by 5.7% and quality by 8.4%
Inadequate risk analysis can lower  productivity by 12.5% and quality by 17.5%
Inadequate testing can lower  productivity by 13.1% and quality by 18.1%
Inadequate measurement of quality can lower  productivity by 13.5% and quality by 18.5%
Inadequate defect tracking can lower  productivity by 15.3% and quality by 20.9%
Inadequate inspections can lower  productivity by 16.0% and quality by 22.1%
Inadequate progress tracking can lower  productivity by 16.0% and quality by 22.5%

N.B. progress tracking is on the list twice.  Informal progress tracking is fairly neutral, but inadequate progress tracking is deadly.

Conclusion

In development practices it is not a matter of formal vs informal.  It is a matter of seeing what kind of variation that you are exposed to and recognizing that formality can make a difference in that area.  Then it is a matter of only adding as much formality as you need to get maximum productivity.

Formal practices must be monitored, you must have some way of testing if the formal practice is having an effect on your development.  You want to make sure that you are executing all formal practices properly and not just going through the motions of a formal practice with no effect.  You must monitor the effects of formality to understand when you are not doing enough and when you are doing too much!

Articles in the "Loser" series
Moo?

Want to see sacred cows get tipped? Check out:
Make no mistake, I am the biggest "Loser" of them all.  I believe that I have made every mistake in the book at least once :-)
Posted by at No comments:
Labels: , , , , , , ,
Subscribe to: Posts (Atom)