Why You Should Avoid Using Projects in Software

Summary

Organizing work around projects does not fit any form of work that is creative, open-ended, and patterns-based, relying on human ingenuity and judgement to create value. This is most notable in research, creative jobs, and development in all its forms; be it organizational change, personal growth or software development.

Projects in Open-ended Endeavours

What would the 15-year-old Jack Nicklaus, the legendary golfer, have responded if his coach had asked him for the probability that he’d meet milestone 3 for project Winning British Open as defined in the project plan as per 7th of March?

Yes it’s kind of silly to imagine this happening – yet in software this is just a part of normal life. Every day, thousands of developers and teams are asked to compare their progress against The Plan and estimate how much is left.

Now, I agree that writing software is not the same as developing your golf skills, but considering the level of craftsmanship and inherent variability in software it’s more appropriate and valuable to me to compare it to other kinds of patterns-based, open-ended activities than with production lines chugging out the same item over and over again.In these endeavours having a plan is actually less important that making steady progress and creating value.

A painting is finished when the artist says so, based on skill and experience gathered over a lifetime. But from the customer’s perspective it’s finished when it is finally destroyed. So who’s right? Both, I guess. A defined finish seems to not exist here. The same thinking can be applied to, for example, a research paper or a software program I think the common element here is that we are creating — and anything created has a life cycle, which resists being put into a project management framework.

Just to be clear, when I write project here I don’t mean a batch of work that belongs together. The latter are naturally occurring “lumps of work”. I use the term initiatives or epics for those. When I talk a about projects I mean a way of organizing work following some kind of formal or informal project model, typically lead by a project manager, with the expressed goal of delivering everything in scope on time and within budget. An initiative doesn’t need any of that.

But What About Agile Projects?

People still ask me: How should we work in agile projects? To start with, anything prefixed with  “agile” is cause for concern. Anything that requires modulation cannot possess that characteristic in itself, by nature, but must be adjusted to fit. That’s why we have “agile requirements”, “agile documentation”, “agile contracts”, “agile planning tools”, “agile processes”, “agile management”, and… “agile projects”.

So we know that projects need to be modified to fit an agile organization. And many people have written about that to explain how you should adjust your project model to work in an agile fashion. That’s fine, but that’s not my point. My point is this: Why bother?

Asking about how to do agile projects is the wrong question I feel, since it assumes that projects are the best (only?) way to work with software. A more interesting questions to me is this: How should we organize our work to create maximum value (learning) over time? And to the this question I strongly believe that the correct answer cannot be “in projects”.

Why Projects Aren’t Suitable In Software

Here’s a list of reasons I’ve compiled over the years, use it as you see fit…

1. Projects encourages important thinking upfront, investigating problems and drawing up solutions, but… we know that we want to harness the learning built into development. Why make so many decisions right at the start, when we know the least?
2. Projects by definition have a defined start and ending, after the project the results must be handed over to some other type of organization, but… software has a life cycle. Software is created, developed, actively maintained, passively maintained, and finally, decommissioned. Software and projects operate on different timescales.
3. Project results are typically handed over to another group for “maintenance”, but… we know that handovers are costly and excellent quality requires ownership and continuity.
4. To reach project goals, especially on dates and costs, quality is pushed to the background. Some projects even result in so-called “death marches”, where delivering something maintainable is not in scope, but… in software, quality is a primary success factor… because without excellent intrinsic quality, your product will be slow and expensive to maintain and have a high life cycle cost.
5. Projects encourage us to create batches of work, being a big batch in itself, but… we know that a steady flow of value is way more effective (feedback + learning).
6. Projects (and PMs) often entertain a narrow view of success, i.e. what’s best for the project, but… wouldn’t we rather optimize for the whole company? A project is a target to a PM and, as W. Edwards Deming famously said: “Give a manager a target and he will reach it, even if he has to destroy the company in the process“. Especially large projects tend to behave a bit like the old schoolyard bullies, taking what they feel like without asking.
7. To acquire funding for their projects, PMs are often tempted or forced to overstate the expected positive outcomes of their projects, otherwise it won’t get funding, ending up in a never-ending spiral of silly overstatements, but… most people are decent and prefer to work in an environment where they don’t need to deceive others to do their job.
8. Projects typically require setup, allocating people, finding resources like office space, writing presentations, holding lift-offs, agreeing on vision, mission, and how to work etc. In most settings you pull people to the project and not the opposite, but… we can do away with all that by simply letting teams already in full operation pull work to themselves instead.
9. Projects use budgetary mechanisms to give attention to ideas and allocate funding to them, but… having funding is a very poor raison-d’être while creating happy customers is a good one.
10. For psychological reasons (for example, ego and careerism) it is tempting to create massive projects for bold ideas, but… we know that large projects are associated with stunningly high risks and low success rates, close to zero after some point.

These observations have nothing to do with agile methods or any other school of software development. In fact, it has nothing to do with software development at all. It’s at the heart of all creative, open-ended work, including arts, craft, writing, architecture or organizational change. This also explains why having an agile “rollout” plan is silly. Change is not a project and it’s not as easy to simply “implement” it (I wish it was).

Now I don’t want to imply that if a client comes up with a project plan you should laugh in their face. Of course not. There are situations where projects can be suitable, e.g. when working with a new contractor team outside your organization. You can use the project to get to know each other and the defined end date to call things off if it didn’t work out. Just be aware of the risks that projects bring and work to mitigate them.

The Alternative

The alternative to not working in projects can be outlined like this:

• Create and grow long-term, cross-functional development teams that create their own, smooth development process.
• Give each team clear ownership of a product, part of a product or a few products, depending on size. Cultivate a long-term, product outlook on development.
• Task each team with delivering value to customers (or learnings if something is unsuccessful) continuously, as often as the market can handle it, while expending less and less energy to do it.
• Help the teams create more value to their customers using sound prioritization schemes and pull systems for work.
• If suitable, batch larger work into initiatives, lasting a couple of months (enough to create milestone challenges to the teams), but keep scope high-level and stay open to changes and complete turnarounds as the learnings start pouring in.

You can of course persist and keep trying to make agile projects work, but seriously, why bother if you don’t have to?

When the manager’s out, the coaches dance on the table

I work as a software consultant and often in the role of a so-called “team coach” or “agile coach”. That’s basically just a hyped-up term for helping teams and departments to improve, to be more effective, and guide them on their way amongst a myriad of options. So coaching is great and many coaches I know are great, but there’s something bothering me. My opinion is that much of what I do is a kind of management – not of the people but of the work, and I’m getting more and more concerned about this. Should this really be the coach’s job?

Why CEOs Behave Like Cowards

“The times they are a-changin'”, as Bob Dylan put it and that’s more true than ever today. Software is eating the world. Every firm is a software company now, whether they understand it or not. We’re in what Steve Denning calls the creative economy, driven by customers with an infinite amount of information at their disposal.

In these times of upheaval you might guess that many companies are going through major internal reflection and restructuring. The general answer is sadly, no. Many companies are still by-and-large acting like product pushers to the ignorant masses. Almost every traditional company is still leaning on tenets of management created in the 1800s. Radical changes are far and few apart. As a result of this fundamental inability to adapt, the life expectancy of firms have dropped significantly, now less than fifteen years and declining rapidly (Fortune 500 companies).

I think it’s fair to say that many existing organisations are in desperate need for major redesign, not just structurally but dynamically, how they work to create value. Nothing happens. It seems to me that CEOs opt for a safe tactic, maintaining the system instead of fundamentally changing it. CEOs are playing not to lose instead of playing to win.

Now this made me curious. I mean, why is that? I mean, are they idiots or just incompetent? Hardly. Perhaps some other forces are at play?

A Latte with Deming: How I learned to stop blaming employees

There is a coffee shop in southern Stockholm where I have taken the habit of procuring my morning shot of coffee. There’s nothing special about the place, devoid of charm and located at the busy entrance of a metro station, but the coffee’s pretty good. I always order a double latte with an additional carrot-and-orange juice. One nice touch is that they put cardboard holders around hot paper cups, making them less uncomfortable to hold.

Photo “Cappuccino coffee with latte art tulip” by Takeaway@Wikimedia.org, available under a Creative Commons Attribution-ShareAlike licence

The coffee shop is managed by a middle-aged man, often working in the back room. The best thing about the place is probably not the coffee, but the young, female shop-assistant, who’s very service-minded and efficient.

This morning routine of mine hummed along nicely until one day a couple of weeks ago when, to my concern, I was met by two girls behind the counter. The new girl looked very confused. During the following weeks, the training period I assume, the new and old assistant worked side by side, but after that, the one I was used to was gone.

Honey, let’s start over: A tale of business kaikaku

When Ian told them what he’d done, they couldn’t believe their ears. He had what!? He had erased every document in the company database concerning hiring and retaining staff. Joan’s deep thoughts on “talent management” simply gone. Gone! “Hey, what about the backups?”, Helen asked. Ian just smiled, “Those too”.

Promises, Promises (2014)

The Pledge of the Sheriff

In the movie The Pledge (2001), directed by Sean Penn, a weary police chief, played by Jack Nicholson, investigates the murder of a young child. In the pivotal scene, on the day of his retirement, he promises the mother of the murdered girl to find the killer.

The sheriff gradually goes to extremes to fulfil his promise. He dedicates all his time to the case. He moves to a small town in the mountains, where he suspects the murderer could be found. The pledge he made propels him to forsake everything else. His mental health starts to decay. The guilt and shame of the unfulfilled promise lie heavily on his shoulders.

This movie reveals how incredibly powerful a promise can be, even today. Human beings can become very preoccupied in their quest to fulfil a promise.

Four Horrible and Eight Good Ideas in a Project Crisis

In my last post I described a project restart, a simple, generic process for handling a software project when there are too many things left to do a few months before deadline. In this post I’d like to get more hands-on and talk about what actions you might actually consider in this unpleasant situation.

My experience is that some ideas that people consider sound great but are really, really bad ideas. And some suggestions, that many people may be unaware of, could have a positive impact.

Be warned, though! There is no magic dust you can sprinkle on a team to make them considerably faster in the short term. If you can help them become just 10-20% more productive you should feel proud. You have more leverage reducing scope on every level and modifying the system in which they work.