Wednesday, 30 November 2011

Why Adding Personnel to a late Software Project delays it more

The previous blog entry explains how poor requirements and insufficient team synchronization mechanisms can lead to constant fire-fighting.  When faced with constant fire-fighting your project starts spinning out of control and code development will slow to a crawl.  At this time, management's first instinct is to throw more developers at the problem.

While adding resources to a late project seems like a logical thing to do, it generally makes the problem worse, i.e. leads to more fire fighting and even slower code production.  While it seems counter-intuitive, actually throwing people off the project is more likely to make your project move faster.  Fred Brooks, author of The Mythical Man-Month calls this principle Brook's law.

Different Types of Team Activity
Before we address why late projects slow down when you add resources, we must first review the different types of team activities and their inherent productivity characteristics.  When teams of people perform tasks they fall into one of three different categories: 1) additive, 2) disjunctive, and 3) conjunctive.

In an additive activity, the productivity of the group is determined by adding up the productivity of each of the individuals comprising the team, i.e. team productivity = (individual productivity)[1].  One additive activity is tug-of-war where the productive output of your team is equal to the sum of the pulling force of all the members of your team.  Another additive activity would be a team of people painting a house. 

Managers throw additional people into late projects on the assumption that coding is an additive activity, it isn’t; we'll cover why in a second.

In a disjunctive activity, the productivity of the group is determined by the strongest member of the team, i.e. team productivity = max(individual1, individual2, …, individualn).  A disjunctive activity would be playing Trivial Pursuit in large teams, if any person on the team gets the answer correct the whole team gets the correct.  In software projects disjunctive activities occur when there is a very specific technical problem to solve.  In the meeting, whoever solves the problem first will solve it for the entire team.

In a conjunctive activity, the productivity of the group is determined by the weakest member of the team, i.e. team productivity = min(individual1, individual2, …, individualn).  Conjunctive activities are equivalent to the weakest link in a chain.  Security is a conjunctive activity, you are only as secure as the weakest part of your security architecture.  Quality is a conjunctive activity and this is why we say "quality is everyone's job".  It only takes one poor quality component to reduce the quality of an entire product.

When an organization is unaware of critical conjunctive activities, they are likely to have all kinds of execution problems.

Understanding Requirements is a Conjunctive Activity
Software projects get into a fire fighting mode because there is a poor understanding of the requirements from a team perspective.  Whether the requirements were well written or not, if those requirements are poorly understood by the team then you start playing 6 blind men and the elephant. 

This is where you discover that everyone in your project has a different perspective on what the system is supposed to do and how it is supposed to do it.  The fire-fighting mode is nothing more than a set of meetings to resolve differences and solve problems caused by divergent beliefs on the project.

Understanding the requirements is a conjunctive activity.  Your productivity is only as good as the weakest understanding in the team.  The developer on the team with the weakest understanding of the requirements is probably generating the most defects.  If QA does not understand the requirements (if they exist) then they will be generating all kinds of false positives when they are unsure the software is behaving properly.

With this perspective, it is easy to see how adding people to a late project will cause it to be later.  The additional developers and QA being added to the project will have the poorest understanding of the requirements of all the team members.  This means that they will almost certainly generate more defects in development and cause even more false positives in QA.  This will increase the amount of fire-fighting that you do and cause the project to slow down even more.

Solution: Throw People off the Ship
Walk the plank
So as counter-intuitive as it sounds, you need to throw people off the ship.  Find the developers and QA personnel who don't understand the requirements and remove them from the project.  These are the guys creating much of the noise in the fire-fighting meetings. 

Otherwise get these people together with the business analysts and educate them about what the software is supposed to do and how it is supposed to be done.  If you are going to add personnel to the team then this becomes an ideal time to get them educated on the requirements BEFORE they start producing or testing code.

While they are not working directly on the project have them put together the centralized requirements repository suggested in the last blog.  Once they are sufficiently familiar with the requirements then you can add them back to the software team.

Additional resource: The Mythical Man Month, by Fred Brooks

[1] ∑ is the Greek symbol used for summation

Tuesday, 29 November 2011

Root cause of 'Fire-Fighting' in Software Projects

Quite a few projects descend into 'continual fire fighting' at the end of a project or software version release.  Suddenly there are an endless set of meetings which involve the product managers (or business analysts), developers, QA, and managers.  Even when these meetings are well run, you cut the productivity of your developers who can barely get a few contiguous hours to write code between the meetings.

Ever wonder what causes this scenario to occur in so many projects? Below we look at the root causes of fire fighting in a project.  We also try to suggest meeting strategies to maximize productivity and minimize developer disruption.
  
The first thing to notice is the composition of the meetings when fire fighting starts.  One common denominator is that it is rarely just the developers getting together to solve a problem involving some technical constraint; often these are cross-functional meetings that involve business analysts and QA.  In larger organizations, this will involve end-users and customers.  Fundamentally, fire-fighting is the result of poor coordinating mechanisms between team members and confused communication.

Common Scenarios that Waste Time
Typically an issue gets raised in a bug triage meeting about some feature that QA claims is improperly implemented.  Development will then go on to explain how they implemented it and where they got the specific requirements.  At this point, the business analyst chimes in about what was actually required.  There are several basic scenarios that could be happening here:

  1. The requirements are complete and QA is pointing out that development has implemented the feature incorrectly.
  2. The requirements are loose and development has coded the feature correctly but QA believes that the feature is incorrect
  3. QA has insufficient requirements to know if the feature is implemented correctly or not.
  4. The requirements are loose and development and QA have different interpretations of what that means.
    Scenario 1 is what you would expect to happen in a bug triage session.  There is no wasted effort for this case as you would expect to need the business analyst, development, and QA to resolve this issue.

    Scenario 2 is what happens when the requirements are not well written.  Odds are the developer has made several voice calls and emails to the business analyst to resolve the functionality of a particular feature.  This information exists purely in the heads of the business analyst and the developer and is buried in their email exchanges and does not make it back into the requirements.  This scenario wastes the developer's time.

    Scenario 3 also happens when the requirements are not well written.  Most competent QA personnel know how to write test plans and test cases.  When the requirements are available to QA with enough time, they can generally determine if they have sufficient information to write the test cases for a given feature.  If given the requirements with enough time, QA can resolve the ambiguity with the business analyst and make sure that the requirements are updated.  When there is insufficient time, the problem surfaces in the bug triage meeting.  This scenario wastes both QA and development's time.
    Scenario 4 occurs when you have requirements that can legitimately be implemented in many different ways.  It is likely that QA did not get the requirements before coding started, if so they could have warned the business analyst to fix it.  If development has implemented the feature incorrectly, then: 1) the business analyst needs to fix the requirement, 2) development needs to re-code the feature, and 3) QA needs to update their test cases.  In this scenario everyone's time is wasted.

    If the your scenarios are not 2), 3), and 4) then you are probably in fire fighting mode because you have requirements that can not be coded as specified due to unexpected technical constraints.  Explaining to the organization why something is technically infeasible can take up quite a few meetings. 

    As an example of unexpected technical constraints, at Way Systems (now Verifone) we were building a cell phone POS system.  Typically signal strength is shown as 5 bars, however, due to the 3rd party libraries we were using we could only display a number 0-32 for the wireless signal strength.  There was no way to overcome the technical constraint because there were too many framework layers that we did not control.  Needless to say there were quite a few (useless?) meetings while we informed everyone about the issue.

    Strategies to Reduce Fire-Fighting
    The best way to reduce fire-fighting is simply to have effective requirements when you start a project.  Once you are caught in fire-fighting the cure is the same – you need to fix the requirements and document them in a repository that everyone has access to.  By improving the synchronization mechanisms between the business analysts, development, and QA your fire fighting meetings will go away.  In particular, all those requirements discussions that the business analysts have had with QA and development need to be written down.

    Centralize and Document Requirements
    If you are using use cases then the changes need to be made in the use case documents.  If you don't have a centralized repository then you need to create one.   You can use a formal collaboration tool such as SharePoint, an informal collaboration tool such as Google Sites, or simply use a Wiki to host and document all requirements.

    In your document repository you will want to keep all requirements by scenario.  If you are using use cases or user stories then each of these is a scenario.  If you have more traditional requirements they you will need to determine the name of the scenarios from your requirements.  Scenarios will be of the form 'verb noun phrase', i.e. 'create person', 'notify customer of delivery', etc.

    Once you have a central repository for putting your requirements then ALL incremental requirements should be put on this site, not in cumbersome email chains.  If you need to send an email to someone, then document the requirement to the central site and email a link to the party; do not allow requirements to become buried in your email server.

    Run Effective Meetings
    Managers are often tempted to call meetings with everyone present 'just in case'.  There is no doubt that this will solve the occasional problem, but you are likely just to have a bunch of developers with 'kill me now' expressions on their faces from beginning to end in the meeting.

    Structure the meeting by grouping issues by developer and make an agenda so that each developer knows the order that they will be at the meeting.  No developer should have to go to a meeting that does not have an agenda! Next, use some IM tool from the conference room to let developers know when they are required to attend the meeting (not the 1st developer, obviously :-) ).  Issues generally run over time so don't call anyone into the meeting before they are really required.  Give yourself breathing room by having meetings finish 10 minutes before the next generally used slot (i.e. 10:20 am or 2:50 pm).

    Issues that really need multiple developers present should be delayed for end of the meeting.  When all other items are handled, use IM to call all the developers for those issues at the end.  By having the group issues at the end you are unlikely to keep them around for a long time since you will probably have to give up your conference room to someone else.

    Conclusion
    Not all fire-fighting involves bad requirements, but many of them do.  By trying to produce better requirements at the start of a project and implementing centralized mechanisms for those requirements you will reduce the fire-fighting later in your project. If you find yourself in fire-fighting mode, you can use implement a centralized requirements mechanism to help fight your way out of the mess.