Why You Should Avoid Using Projects in Software

Summary

Organizing work around projects does not fit any form of work that is creative, open-ended, and patterns-based, relying on human ingenuity and judgement to create value. This is most notable in research, creative jobs, and development in all its forms; be it organizational change, personal growth or software development.

Projects in Open-ended Endeavours

What would the 15-year-old Jack Nicklaus, the legendary golfer, have responded if his coach had asked him for the probability that he’d meet milestone 3 for project Winning British Open as defined in the project plan as per 7th of March?

Golf ball 3

Yes it’s kind of silly to imagine this happening – yet in software this is just a part of normal life. Every day, thousands of developers and teams are asked to compare their progress against The Plan and estimate how much is left.

Now, I agree that writing software is not the same as developing your golf skills, but considering the level of craftsmanship and inherent variability in software it’s more appropriate and valuable to me to compare it to other kinds of patterns-based, open-ended activities than with production lines chugging out the same item over and over again.In these endeavours having a plan is actually less important that making steady progress and creating value.

A painting is finished when the artist says so, based on skill and experience gathered over a lifetime. But from the customer’s perspective it’s finished when it is finally destroyed. So who’s right? Both, I guess. A defined finish seems to not exist here. The same thinking can be applied to, for example, a research paper or a software program I think the common element here is that we are creating — and anything created has a life cycle, which resists being put into a project management framework.

Just to be clear, when I write project here I don’t mean a batch of work that belongs together. The latter are naturally occurring “lumps of work”. I use the term initiatives or epics for those. When I talk a about projects I mean a way of organizing work following some kind of formal or informal project model, typically lead by a project manager, with the expressed goal of delivering everything in scope on time and within budget. An initiative doesn’t need any of that.

But What About Agile Projects?

People still ask me: How should we work in agile projects? To start with, anything prefixed with  “agile” is cause for concern. Anything that requires modulation cannot possess that characteristic in itself, by nature, but must be adjusted to fit. That’s why we have “agile requirements”, “agile documentation”, “agile contracts”, “agile planning tools”, “agile processes”, “agile management”, and… “agile projects”.

So we know that projects need to be modified to fit an agile organization. And many people have written about that to explain how you should adjust your project model to work in an agile fashion. That’s fine, but that’s not my point. My point is this: Why bother?

Asking about how to do agile projects is the wrong question I feel, since it assumes that projects are the best (only?) way to work with software. A more interesting questions to me is this: How should we organize our work to create maximum value (learning) over time? And to the this question I strongly believe that the correct answer cannot be “in projects”.

Why Projects Aren’t Suitable In Software

Here’s a list of reasons I’ve compiled over the years, use it as you see fit…

  1. Projects encourages important thinking upfront, investigating problems and drawing up solutions, but… we know that we want to harness the learning built into development. Why make so many decisions right at the start, when we know the least?
  2. Projects by definition have a defined start and ending, after the project the results must be handed over to some other type of organization, but… software has a life cycle. Software is created, developed, actively maintained, passively maintained, and finally, decommissioned. Software and projects operate on different timescales.
  3. Project results are typically handed over to another group for “maintenance”, but… we know that handovers are costly and excellent quality requires ownership and continuity.
  4. To reach project goals, especially on dates and costs, quality is pushed to the background. Some projects even result in so-called “death marches”, where delivering something maintainable is not in scope, but… in software, quality is a primary success factor… because without excellent intrinsic quality, your product will be slow and expensive to maintain and have a high life cycle cost.
  5. Projects encourage us to create batches of work, being a big batch in itself, but… we know that a steady flow of value is way more effective (feedback + learning).
  6. Projects (and PMs) often entertain a narrow view of success, i.e. what’s best for the project, but… wouldn’t we rather optimize for the whole company? A project is a target to a PM and, as W. Edwards Deming famously said: “Give a manager a target and he will reach it, even if he has to destroy the company in the process“. Especially large projects tend to behave a bit like the old schoolyard bullies, taking what they feel like without asking.
  7. To acquire funding for their projects, PMs are often tempted or forced to overstate the expected positive outcomes of their projects, otherwise it won’t get funding, ending up in a never-ending spiral of silly overstatements, but… most people are decent and prefer to work in an environment where they don’t need to deceive others to do their job.
  8. Projects typically require setup, allocating people, finding resources like office space, writing presentations, holding lift-offs, agreeing on vision, mission, and how to work etc. In most settings you pull people to the project and not the opposite, but… we can do away with all that by simply letting teams already in full operation pull work to themselves instead.
  9. Projects use budgetary mechanisms to give attention to ideas and allocate funding to them, but… having funding is a very poor raison-d’être while creating happy customers is a good one.
  10. For psychological reasons (for example, ego and careerism) it is tempting to create massive projects for bold ideas, but… we know that large projects are associated with stunningly high risks and low success rates, close to zero after some point.

These observations have nothing to do with agile methods or any other school of software development. In fact, it has nothing to do with software development at all. It’s at the heart of all creative, open-ended work, including arts, craft, writing, architecture or organizational change. This also explains why having an agile “rollout” plan is silly. Change is not a project and it’s not as easy to simply “implement” it (I wish it was).

Now I don’t want to imply that if a client comes up with a project plan you should laugh in their face. Of course not. There are situations where projects can be suitable, e.g. when working with a new contractor team outside your organization. You can use the project to get to know each other and the defined end date to call things off if it didn’t work out. Just be aware of the risks that projects bring and work to mitigate them.

The Alternative

The alternative to not working in projects can be outlined like this:

  • Create and grow long-term, cross-functional development teams that create their own, smooth development process.
  • Give each team clear ownership of a product, part of a product or a few products, depending on size. Cultivate a long-term, product outlook on development.
  • Task each team with delivering value to customers (or learnings if something is unsuccessful) continuously, as often as the market can handle it, while expending less and less energy to do it.
  • Help the teams create more value to their customers using sound prioritization schemes and pull systems for work.
  • If suitable, batch larger work into initiatives, lasting a couple of months (enough to create milestone challenges to the teams), but keep scope high-level and stay open to changes and complete turnarounds as the learnings start pouring in.

You can of course persist and keep trying to make agile projects work, but seriously, why bother if you don’t have to?