Friday, 31 May 2013

Agility is not Informality

Agile Formality:A 
 Mole
” of Software Engineering Practices
Agility is about following the Manifesto for Agile Software Development, not the sloppy and crazy implementations of Scrum, XP, or other home-grown schemes out there.

Sometimes I feel like people are using Agility as an excuse to be informal. Two of the key principles of the Agile Manifesto are:
  • Working software over comprehensive documentation
  • Responding to change over following a plan
Somehow some developers have interpreted this as meaning that there are no formal processes to be followed. But just because you are putting the priority on working software does not mean that there can be no documentation; just because you are responsive to change doesn't mean that there is no plan.

Agile development when properly implemented can accomplish great things.  When it is just an excuse to be undisciplined and cover-up cowboy development then a so called 'Agile' process will work very poorly and the organization will just spin its wheels.

Agile development in general raises productivity by 18.2% and quality by 27.1%

Extreme programming (XP) specifically raises productivity by 12.8% and quality by 17.8%

Agile software development does have formality and it requires quite a bit of discipline to stick with it. A few formal practices include:
  • Stand-up meetings
  • Work burn down charts
  • Sprint demos
  • Simplicity
  • Improvement through reflection
Stand-up Meetings
Stand-up meetings are an integral part of the Scrum process and is a key discipline of Scrum, when they are done properly. This meeting is a coordinating mechanism of a self-organizing team where each member states:
  • What have they finished
  • What are they working on
  • What external issues do they need help on
The scrum daily meeting raises productivity by 16.4% and quality by 23.5%

Believe it or not, there are some Scrum teams that sit down for the stand-up meeting. What the heck is this? The purpose of standing up is to be slightly uncomfortable and keep in mind that the meeting is supposed to move quickly.

You state what features that you have finished so that team mates waiting on your deliverables know where you are at. Finished means that you have unit tested your code (i.e. TDD) and that black box testing can start if there is a tester in your Scrum team.

Automated unit testing raises productivity by 16.5% and quality by 23.8%

You state what you are working on so that other team mates know what you are doing and can organize their work accordingly.  Don't bother to state how you solved problems, that was an issue for planning with team mates before you started coding.

You state what you need help on that is external to the Scrum team so that the Scrum master can resolve the issue.  This is not a design meeting, if there are issues that you need to resolve with team mates then you can resolve this issue right after the meeting.

Burn Down Charts
Work burn down charts is a formal discipline and essential to success (see Who Needs Formal Measurement?).  The work burn down chart is the heart of accountability in Scrum.  For the chart hours to get to 0 on a regular basis, it relies on the principles of: 1) commitment, 2) execution, and 3) estimates.

The point of a self-organizing team is that it commits to work each sprint to get the work done.  If it turns out that they over committed to a sprint then they must do whatever it takes to get the job done.  That may mean extra hours or it may mean being extra creative, but once you commit to work then you have got to get it done.  That means execution, it means doing what you say that you will do.

You may have a sprint or two where the work burn down chart does not go to 0.  If this is happening on a regular basis then you have either an estimation problem or a team commitment problem.  If engineers are regularly underestimating their tasks then it means that you really don't know how long things take.  You should consider some of the techniques in the Personal Software Process (PSP) to improve your ability to estimate.

PSP development raises productivity by 21.3% and quality by 31.3%

Another cause of work burn down charts not going to 0 is that the team is not really self-organizing.  The team needs to commit to the work to be done, it will never work if the organization assigns the work to be done in the sprint.

If you lose discipline during the sprint and don't keep the work burn down charts up to date, then you are not tracking progress correctly.  Remember, you can't manage what you don't measure.

Inadequate progress tracking reduces productivity by 16.0% and quality by 22.5%

Sprint Demos
The sprint demo is not optional, it is a mandatory part of the process.  The sprint demo is to demonstrate near-production code and demonstrate a working system.  The team needs to put pressure on itself that the code must be complete by the end of the sprint; this forces a self-organizing team think through the amount of work that they can commit to.  No demo = no team commitment.

Remember the whole point of Agile development is to put the emphasis on working software.  If you are not producing working software every sprint, then what are you doing? Whatever you are doing it is not Agile development.

The Agile Manifesto puts emphasis on working software.  If your sprints are not producing working software then you are not doing agile development.

Simplicity
The best designs are the simplest designs because they are easier to understand and maintain.  In a previous article No Experience Required! I discuss how years of experience do not make for better programmers.  You might expect that the best developers have more years of experience, but 8 studies over 30 years show that this is not the case (you might even want to get rid of poor performers with many years of experience, but see the article :-) )

In fact two persistent facts about the differences between the best and worst developers is that:
  • program execution speed about 10 to 1
  • program size 5 to 1
Much of this results from simplicity. The best developers plan their code and write simple and straight forward routines. The are rewarded by having smaller programs that execute quickly. The worst developers do not plan their code and shoot from the hip. They write convoluted code that is much larger and has much worse execution time.

Agility is about simplicity. It is spending 80% of your time thinking and 20% of the time writing simple and elegant code. It is not about spending 100% of your time writing complex and convoluted code that even you can't understand.

Simplicity does not happen, it is planned; it takes tremendous effort to write simple code.  The best developers plan their code (see PSP above) and are continually refactoring their code to make designs simpler to understand and maintain.

Automated restructuring raises productivity by 8.0% and quality by 11.7%

Code refactoring raises productivity by 4.3% and quality by 6.7%

Improvement through Reflection
A key discipline in Agile development is to reflect on what you can do better as individuals and as a team. There is always room for improvement, there is often a better way to do things.  Professional developers are keen to keep learning and improving their craft, and the best way to do this is by reflecting on the sub-optimal choices that we have made in the past and trying to do better in the future.

Conclusion
Agile development is about following the Agile Manifesto, it is not about informality.  True agility is developed by having the discipline to be formal in many things:
  • Team synchronization
  • Learning to estimate
  • Tracking progress through burn down charts
  • Committing to create production code
  • Planning simple code
  • Improving by not making the same mistakes
Agility is about being disciplined and formal in processes that count


References
1N.B. All productivity and quality percentages were derived over 15,000+ actual projects
2Reference courtesy of Michael Jessop

Articles in the "Loser" series
Moo?

Want to see sacred cows get tipped? Check out:
Make no mistake, I am the biggest "Loser" of them all.  I believe that I have made every mistake in the book at least once :-)