No Experience Required!

Did you know that we have never found a relationship between a developer's years of experience and code quality or productivity?

The original study that found huge variations in individual programming productivity was conducted in the late 1960s by Sackman, Erikson, and Grant (1968).  

This study has been repeated at least 8 times over 30 years and the results have not changed! (see below) 

Sackman et al studied professional programmers with an average of 7 years’ experience and found that:

• the ratio of initial coding time was about 20 to 1
• the ratio of debugging times over 25 to 1
• program execution speed about 10 to 1
• program size 5 to 1

They found no relationship between a programmer's number of years of experience and code quality or productivity.  That is there was NO correlation between experience and productivity (i.e. ability to produce code) and there was NO correlation between experience and quality (i.e. minimizing defects) .

Think about that for a minute...

That is the worst and the best programmers made distinct groups and each group had people of both low and high experience levels.  Whether training helps developers or not is not indicated by these findings, only that years of experience do not matter in the aggregate.  If training does matter then very few developers are getting trained.

Without considering legality, it is simpler to get rid of expensive poor performers and hire good performers with few years of experience!

This does not mean that training does not make a difference for some people.  What it means is that unproductive people tend to stay unproductive over their entire career.

Results Have Been Confirmed for 30 Years!

There were flaws in the study, however, after accounting for the flaws, their data still shows more than an order of magnitude difference between the best programmers and the worst, and that difference was not related to experience.

In years since the original study, the general finding that "There are order-of-magnitude differences among programmers" has been confirmed by many other studies of professional programmers (full references at the end of the article):

• Curtis 1981
• Mills 1983
• DeMarco and Lister 1985
• Curtis et al. 1986
• Card 1987
• Boehm and Papaccio 1988
• Valett and McGarry 1989
• Boehm et al 2000

Technology is More Sophisticated, Developers are not

You might  think that we know much more about software development today since 1968, after all today: 

• we have better computer languages
• we have more sophisticated technology
• we have better research on effective software patterns
• we have formal software degrees available in university

It turns out that all these things are true, but we still have order of magnitude differences among programmers and the difference is not related to years of experience.  

That means that there is some other x-factor that drives productive developers;  that x-factor is probably the ability to plan and make good decisions.

The bad news is that if you are unproductive then years of experience won't make you better

Developers face making decisions on how to structure their code every day.  There is always a choice when it comes to:

• laying out code pathways
• packaging functions into classes
• packaging classes into modules

Because developers face coding complex decisions, the best developers will plan their work and make good decisions.

Bad developers just 'jump in'; they assume that they can always rewrite code or make up for bad decisions later. Bad developers are not even aware that their decision processes are poor and that they can become much better by planning their work.

When have you gotten time to go back and fix suboptimal decisions?

Solution hinted at by Watts Humphrey

Watts Humphrey tried to get developers to understand the value of planning development, and making decisions in the Personal Software Process (PSP) for individuals and the Team Software Process (TSP) for teams. An analysis of over 18,000 projects demonstrates that¹:

PSP can raise productivity by 21% and quality by 31%

TSP can raise productivity by 21% and quality by 31%

This can have a profound effect on the way that we build and train our teams.  Rather than having large teams of mediocre developers, it makes much more sense to have smaller teams of highly productive developers and train them on how to plan and make good decisions.  

The findings suggest that the best way to rehabilitate a poor developer is to teach them how to make better decisions.

Conclusion

We inherently know this, we just don't do it.  If the senior management of organizations only knew about these papers, we could make sure that the productive people get paid what they are worth and the non-productive people could seek employment in some other field.  This would not only reduce the cost of building software but also increase the quality of the software that is produced.

Unfortunately, we are doomed to religious battles where people debate methodologies, languages, and technologies in the foreseeable future.  The way that most organizations develop code makes voodoo look like a science!

Eventually we'll put the 'science' back in Computer Science, I just don't know if it will be in my lifetime...

Related Articles

• Productive Developers are Smart and Lazy
• Are You are Surrounded by Idiots? Unfortunately, You Might be the Idiot...

Bibliography

Boehm, Barry W., and Philip N. Papaccio. 1988. "Understanding and Controlling Software Costs." IEEE Transactions on Software Engineering SE-14, no. 10 (October): 1462-77.

Boehm, Barry, et al, 2000. Software Cost Estimation with Cocomo II, Boston, Mass.: Addison Wesley, 2000.

Card, David N. 1987. "A Software Technology Evaluation Program." Information and Software Technology 29, no. 6 (July/August): 291-300.

Curtis, Bill. 1981. "Substantiating Programmer Variability." Proceedings of the IEEE 69, no. 7: 846.

Curtis, Bill, et al. 1986. "Software Psychology: The Need for an Interdisciplinary Program." Proceedings of the IEEE 74, no. 8: 1092-1106.

DeMarco, Tom, and Timothy Lister. 1985. "Programmer Performance and the Effects of the Workplace." Proceedings of the 8th International Conference on Software Engineering. Washington, D.C.: IEEE Computer Society Press, 268-72.

¹Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.

Mills, Harlan D. 1983. Software Productivity. Boston, Mass.: Little, Brown.

Valett, J., and F. E. McGarry. 1989. "A Summary of Software Measurement Experiences in the Software Engineering Laboratory." Journal of Systems and Software 9, no. 2 (February): 137-48.