What is software architecture?

What is software architecture? In the previous article "What is architecture?" we stated that architecture is about:

• Structural elements
• Connective elements

Let us talk about these with regard to a software system.  In our next article we'll talk about good and bad architecture.

Structural elements

Structural elements in software are layers that support other layers in the system:

• the language libraries are built on the functionality provided by the O/S
• the functionality of the software product is built on the language libraries
• 3rd part libraries that you are using are built on the previous 2 layers
• your code is built on all the previous layers

So any program you build has at least the following structural elements:

In a well designed system, your code will also break into layers specific to the application that you are building.  Each layer ideally only depends on layers underneath it, otherwise you will have circular references; asignal that there is a challenge or that something is wrong. 

Connective elements

Every software system has connective elements that communicate across the different layers.  Often 3rd party libraries are shared services that provide core services like data structures (XML, JSON, etc), logging, debugging, and other services.

Inside your code you will also have shared services that are used by every layer, this is diagrammed as follows:

Where layer 1, 2, ..., N represent the layers in your code and the share services are shown vertically as they cut across all the layers.  Layer N represents the UI layer of the system and the functionality at that layer depends on all the layers below it.

Generalizing Architecture

When people are talking about software architecture, they are not just talking about the structure of the code.  Often people are talking about the machines and O/S components as well.  So for example when people are talking about the LAMP architecture they are talking about:

• The O/S -- i.e. Linux
• The web server -- Apache, running on Linux
• The database -- MySql, running on Linux,
• The code -- PHP running on the MySql database using services from the O/S

Note: The PHP layer is further broken down like the diagram above in the Structural Elements section.

The LAMP architecture involves structural elements.  Connective elements are simply the ones created inside the PHP layer or generally provided by the O/S.

So when relevant, architecture includes:

• The O/S
• All support services, i.e. webserver, database, etc.
• The high level applications written by you or a 3rd party

General Software Architecture

For the structural elements of software we are taking about layers.  Those layers can be relative to the O/S such as how the O/S communicates with support services (web servers, databases, etc) or how your applications communicate with support services and the O/S.

Within your application there are different layer interactions, there is 1) how your base layers are built on the APIs of your language and the 3rd party libraries that you have installed, and 2) how you have broken up your layers to communicate with the different parts of your software.

The main structural elements of any model-view-controller (MVC) program looks somewhat like this:

• The model layer a.k.a. business layer is built on 3rd party APIs and O/S APIs
• The view layer a.k.a. GUI layer is built on 3rd party GUI APIs
• The data layer is build on database APIs

All of the APIs that you are using are based on the standard APIs of the language that you are using.

Implications

• Changing languages requires that every layer above it be reconsidered
• This means that if you change languages and some 3rd party API must be changed then the layer above also needs to be changed
• Strict separation in layers means that you can change a 3rd party API and only a single layer of the MVC system is affected
• Often developers color between the lines and a change of a 3rd party library causes quite a bit of pain as the layer infractions are found
• Not having enough of a business layer will be problematic because it is one of the main separating structures between your view/GUI and data layers.
• People discover lacking business layers when they change their database APIs or databases and have the problems propagate all the way through the GUI
• Often when business functionality is built into the model/GUI layer then that functionality is lost and needs to be recoded when a new GUI API is used

Connective elements

Connective software elements are used to shuttle data between the structural layers of the system. These connective elements are generally available everywhere (although they can be limited to a few layers).

Easily understood connective elements are those used in debugging and logging.  These elements are available universally, e.g. debugging and logging routines can be made available in all layers without worrying about layer dependency.

Other connective elements are most easily recognized because they are routines shared across layers of the architecture.  More specifically, they are the libraries/modules that are included into multiple layers of the program.

---

So now we have defined what structural and connective elements are in software, next we describe what it means to have good architecture and bad architecture.

Why Outsourcing Fails

Every year, we see corporations outsource operations only to pull back back some, if not all, of the operations. When this happens, the cost of pulling back operations or splitting things over multiple countries can leave you with significant operational issues and similar or worse costs.

Successful corporations understand the need to control costs in operations. The "spare no expense" philosophy of John Hammond in Jurassic Park is a one way ticket to disaster.

But, a focus on cost reduction is what leads corporations to face-fault when they outsource.

Corporations are seduced by the idea of potentially cutting costs dramatically; why pay expensive resources to do repetitive things when overseas people can do the same work for pennies?

The cardinal rule when outsourcing is:

Only operations that you completely understand and control can be outsourced 

To understand operations is to understand the costs of exceptions as well as how they are processed and escalated. Local resources deal efficiently with exceptions because they have informal relationships in the organization and use them to get things done efficiently; those relationships are not available to outsourced resources.

When operations gets outsourced, not only do foreign employees not understand how to deal with exceptions but also:

• They are often in another time zone 
• They don't have full access to local resources 
• They don't know who to contact inside the corporation 
• They don't understand local business rules 
• The network configuration can make it impossible to communicate properly 

Understanding an operational function means that you capture all details, including exceptions, in your information systems. Often, legacy information systems capture exception information as unstructured fields (i.e. memo fields) and off shore resources will not understand this.

99% of your operations might be straight forward, but the 1% of exceptions can at best lead to longer implementation times.  Best case scenario is that you will look bad to your customers and at worst, it will kill your bottom line.

If your information systems contain exception information in a structured way then at least you can eventually train foreign resources to resolve them. But unless exception information is captured so you can control it (i.e. not in general text fields) then it is the same thing as not having it.

When outsourced resources are under strict performance guidelines, they will do anything to clear their work queues. This often involves kicking back work they perceive as an exception as improperly specified; they will claim 'garbage in, garbage out'.

This will result in miscommunications and longer lead times to implement your services. Not only will delivery times, and errors go up, and your customers will get mad.

By the time you have fired the local employees, all the knowledge of the function has left the building and you are left with a broken system.

Key questions to get answers to:

• What does each exception in operations cost? 
• How often do they happen? 
• Can the information systems record exceptions in a way that you can track and control them? 
• Are the mechanisms that local resources use to resolve exceptions be used by outsourced resources? 

If you don't have a clear answer to each of these questions then outsourcing is likely to be a disaster. When you have a clear answer to the 3 questions and understand how you will address each exception then you will have much more success outsourcing.

Schedule Risk is a Red Herring!!!

We often hear the term schedule risk, however, it is generally a Red Herring. Stating that the schedule might stretch is about as useful as saying that eating can cause you to gain weight.

You may be correct but it gives you no leverage to solve the problem

Schedules slip as a result of a problem, if you want to solve the problem then you must identify the root cause.  Any problem will result in a task taking longer than expected and potentially affecting the schedule.

Risk and uncertainty are two sides of the same coin. Without uncertainty there is no risk.

No Uncertainty = No Risk

A risk is a contingent liability; a risk is a future event that is uncertain that has consequences. The key words are future and uncertain.

If 6 months remain and the deadline is in 2 months then there is no schedule risk because there is no uncertainty.

6 months late means that the earliest that the critical path items can finish is in 6 months. Just because the project has not hit the deadline and senior staff doesn't understand the project is late does not qualify the team to talk as if the outcome is uncertain.

It is disingenuous and cowardly to suggest to senior staff that a deadline is possible when you know that it is not.

When the team knows that they are late, they often talk about tasks as being risky simply because they hope that miracles can happen¹.

Hope is not a strategy

In fact, Kahneman points out all of us are wired to bet (pray?) on unlikely outcomes when faced with certain losses, i.e. we double down when faced with a loss.  Team members know about the negative consequences of failure and make projects seem possible simply because they want to delay the pain. Even worse, as the situation gets more desperate people will take bigger and bigger risks.

Using the term schedule risk when a project is not feasible essentially robs the managers of making a course correction until the point where very little can be done.

At a minimum, money can be saved by winding the project down. Few people have the intestinal fortitude to speak out when they know that a project is late. Unfortunately, cowardice is very common.

If you take a paycheck then you have an obligation to your organization to tell them when a project is late.

So it makes no sense to talk about schedule risk when:

• The project is late and you know it
• The project is not late but you see schedule items slipping

In the latter case you are much better to talk about why things are slipping rather than using the term schedule risk.  By talking about the root cause of the slippage, especially early in a project, can lead to you either solving the problem or adjusting the project deadline.  Either way, you will have a greater chance of ending up with a feasible project.

Related Articles

• Why Management Declared Deadlines Lead to Disaster
• No Business Case == Project Failure

References

• Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.
• Jones, Capers. The Economics of Software Quality. 2011
• Kahneman, Daniel. Thinking, Fast and Slow. 2011

When BA means B∪ll$#!t Artist

BA  means Business Analyst, but what makes for a good BA?  When do you have a good BA and when do you have someone who isn't? Many projects fail at the beginning due to incomplete, inconsistent, and overly verbose analysis that then leads to incorrect project plans and projects heading in the wrong direction.

Business analysis consists of all facets of solving business problems. Business analysis is a role executed by project, product, and engineering managers as well as other people.  However, there are people that do business analysis as their main role and we simply label them as business analysts or product managers.

We shall stick to the term business analyst for simplicity. Business analysts perform a range of tasks including:

• Gathering requirements
• Writing business cases and project charters
• Performing gap analyses to incorporate new products

Are your BAs any good?  

The success of many projects depend on the ability of the BA do correctly identify the problem you are trying to solve.  If your BA is not competent then you are doomed before you start.

Ever suspect that the people responsible for performing business analysis for you are not up for the challenge?  Here are three simple questions; good business analysts will get all three correct and not take longer than 30 seconds.

	Context	Question
#1		Which of these two lines is longer?
#2		What does this sign say?
#3	I have two products whose prices add up to $1.10 and one product is $1 more than the other.	How much is the more expensive product?

The answers are:

1. The lines are the same length
   • Take a ruler if you are not convinced
2. Paris in the the spring
   • Notice that the occurs twice
3. The more expensive product is $1.05
   • If the more expensive one had been $1 then the cheaper one would be $0.10 but then the expensive one would only be $0.90 more expensive than the cheaper one.

These three questions illustrate that the mind can jump to conclusions that are often wrong. The mind can be tricked because it assesses things quickly, i.e. jumps to conclusions.  Only some people have the instinct to double check their results and check for personal bias.

Most BAs are well educated; however, the interesting thing is that studies show that educated people are less able to see their biases and they jump to conclusions more readily than other people.¹ Even worse, well educated people have less of a tendency to check their conclusions than other people.

A good business analyst realizes that during analysis there will be situations where the mind will jump to conclusions.  Many business analysts are asked to resolve conflicting requirements, recognize missing requirements, and deal with biases coming from many different sources.  Unless you have the reflex of checking facts for consistency and eliminating bias then you won't make a good business analyst.

So what is the quality of your BAs?

Other articles

• No Business Case == Project Failure
• No Experience Required

Bibliography

¹ West, Richard F. and Meserve, Russel J. Cognitive Sophistication does not Attenuate the Bias Blind Spot. Journal of Personality and Social Psychology.

Stupid is as stupid does

Senior management does not set out to have failed projects, yet 7 out of 10 projects fail and you wonder why they keep doing things the same way.

As Forrest Gump's mother stated, "stupid is as stupid does", that is, smart people sometimes do stupid things.  Most senior managers are pretty smart, however, managers end up doing some very stupid things.

Now from a human perspective, something very interesting is going on here.  For example, suppose you were give the following choices:

1. An 80% chance of making $100
2. A guaranteed $50

Choosing the 1st alternative has an expected value of $80, so people might be expected to choose this because $80 is more than $50. However, because humans are risk averse, almost everyone will choose the second alternative with guaranteed money.

Now out of 10 software projects:

• 3 will succeed
• 4 will be challenged
• 3 will outright fail

To put that in perspective, if you were watching people cross the street at an intersection:

• 3 cross the street successfully
• 4 get maimed
• 3 get killed

How interested would you be in crossing that street?

Stupid is as stupid does...

You can Google "software project failure rates" to see that this has been demonstrated reliably over many industries over 50 years.  Challenged projects are over budget and under deliver but senior management manages to sell them as victories.  I do not consider challenged projects to be successful; hence the statement that 7 out of 10 projects fail.

So if human beings are risk averse and the odds of project success are so low then:

Why does senior management ignore risks on software projects?

It seems likely that senior management can't conceive of their projects failing. They must believe that every software project that they initiate will be successful, that other people fail but that they are in the 3 out of 10 that succeed.

This inability to understand the base rate of failure in software development is systemic. There are so many software projects that are started by senior management where the technical team knows that the chance of success is 0% from the start.

Senior management is human (seriously, they are :-) ) and is risk averse, you just need to find a way to remind them of this. One way to get senior management to think twice about projects is to make sure that there is a meeting before launching the project where management is asked the following question:

Assume that this project will fail, why would it have failed? What will the consequences be?

This exercise (if done seriously) may have the effect of causing senior management to realize that the project can indeed fail. With luck, the normal risk aversion that every human being is endowed with will kick in and the project may get re-evaluated.

Stupid is as stupid does...

Related Articles

• Why Management Declared Deadlines Lead to Disaster
• Stop It!  No... really stop it
  • Article about the 5 most common worst practices in software
• Shift Happens
  • Why scope shift is inevitable in software projects

Bibliography

Kahneman, Daniel. Thinking, Fast and Slow.

Kahneman explains why it is human nature to ignore base rates of failure and why we over-estimate our chances of success in any venture.