The Problem with Software Development is...

Folks like to say that there's a "Software Crisis". We can't build software quickly enough, cheaply enough or well enough.

I agree with EWD -- software is really very, very complex. See EWD 316 for more justification of this position.

Is my conclusion is that the cost of software stems from complexity? Hardly news.

No, my conclusion is that the high cost of software comes from doing the wrong things to manage the high cost of software.

The Illusion of Control

Nothing gives a better illusion of control than a project plan. I think that software development project management tools -- MS Project specifically -- is the biggest mistake we can make.

As evidence, I look at Agile methods. One key element of Agile methods is to reduce (or eliminate) the project management nonsense that accumulates around software development.

I think that software development projects are generally pretty complex and a big MPP file doesn't reduce the complexity or help anyone's understanding. I think that we should not make an effort to capture the complexity -- that's simply silly.

If you find that you need a really complex document to capture a lot of really complex complexity, you're doing something wrong.

Hands in the Pocket Explanations

I think that user stories are great because they reduce the complexity down to something that we can articulate and remember. This gives us a fighting chance at understanding.

If the use case requires a big, complicated document, we're missing something essential. It should have a pithy, easy-to-remember, easy-to-write-on-a-sticky-note summary. It can have a detailed technical appendix. But it has to have a pithy, easy-to-articulate summary.

If you can't explain the use case with your hands in your pockets, it's too complex.

Architecture

An architecture diagram is helpful. Architecture -- as a foundation -- has to be subject to considerable analysis to be sure it's right. You need to be absolutely confident that it works. And like any piece of mathematical analysis, you need diagrams and formulas, and you need to show your work.

A miraculous pronunciation that some architecture will work is a terrible thing. A few pithy formula (that we can remember) and some justification are a whole lot better.

The WBS Is The Problem

I find that projects with complicated WBS's have added a layer of complexity and management that aren't helpful. The cost of software is high, so lets add management to try and reduce our costs. On the surface, adding labor to reduce labor doesn't make sense.

Rather than waste time adding work, it would be better to introduce someone who can facilitate decision-making (i.e., a Scrum Master) and keep progress on track.

Incremental releases of partial solutions have more value than weekly status reports.

Meetings with product owners have more value than a carefully-written schedule for doing the poorly-understood process of detailed design.

Justifications

We can justify project management by saying that it somehow makes the software development process more efficient by eliminating "roadblocks" or "inefficiencies".

I used to believe. I no longer buy this.

Let's look at some candidate roadblocks that a project management might smooth out.

• User Involvement. Or rather, the lack of user involvement. I don't see how a PM does anything except nag the users. If the users aren't motivated to help with software development by answering questions or reviewing the results of a sprint, then the software isn't creating any value. Stop work now and find something the users really want.
• Technical Resources. Coordinating technical resources (DBA's, sysadmins, independent testers, etc.) doesn't require a complex plan, status meetings or reports. It only requires some phone calls among the relevant folks. Directly.
• Decision-Making. The PM isn't the product owner, nor are they a user, nor are they technically proficient enough to understand what's really at stake. Essentially, they only act as a facilitator in a conversation that don't fully understand. That's fine, as long as they stick to facilitating and don't take on responsibilities that aren't assigned to them.

At this point, I can find a use for a facilitator ("Scrum Master"). But I can't see why we have just an emphasis on IT project management. The Agile folks seem to have it right. Reduce cost and complexity by actually reducing the cost and complexity. Not by adding management.