29 January 2010

Programming Is All Design

Posted in Agile, Software development tagged , , at 21.50 by Joakim Holm

There is one, Big Misunderstanding underlying a lot of problems in software development today. This misunderstanding is the reason why programmers are treated as if they were orcs, why business guys have such a hard time collaborating with IT projects, and why outsourcing of software development is generally considered to be a truly great idea. Here it is:

The Big Misunderstanding

Writing software = Building programs.

Most people, even in IT, believe this analogy to be right and good. If you do, then the following statements should also seem basically correct, since they are all based on that view:

  • Customers don’t have to talk to everybody. They should confine their communication to the smart guys. We’ll call them architects. These are the guys customers talk to during project inception to get the details flushed out. After this, the architects take over and handle matters.
  • We should focus on getting the blue prints (the plan, the model, the spec) correct from the start. Let the architects draw diagrams and write specifications for the programmers to implement later.
  • Building programs is a conversion process from drawings and specifications to code. A programmer’s job is to perform this conversion as accurately as possible.
  • Programming is mainly manual labour. The more programmers and keyboards we have, the quicker the implementation will be.
  • We should monitor the programmers to make sure they don’t deviate too much from the specification we have painstakingly created. They should simply build the programs accordingly and all will be fine.
  • If a programmer should find an error (not likely), she would have to prove it to be an error before we can act on it. Changes cost a lot of money, something a programmer probably does not see or care about.

Maybe I’m overdoing it, but I hope that some or even all of these statements sound familiar. If they do, it’s all because of the Big Misunderstanding. They’re all connected and they’re all wrong. Fundamentally wrong!

Where Did These Ideas Come From?

Software development is often compared to the construction industry. Precisely why we admire that field so much is a little hazy. My theory is that it’s a version of something coders call Feature Envy. We want so desperately software development to be considered a serious field, like civil engineering, and what could be more big, manly and serious than being (like) a civil engineer, blowing up stuff and erecting towers. So we ended up with “architects” that produce schematic drawings of software and construction workers (programmers) “building” the software exactly from specification.

But what if the assumption that software development is similar to civil engineering is wrong? What if the implications of engineering lead us to a warped picture of what software development really is? What if this is misleading and generates problems instead?

To determine the truth we need to re-check our assumptions of the nature of software.

Design Targets

The artifacts delivered by a building architect are, more or less, unambiguous and specific. Yes, there is room for the occasional discussion, but the main job of the building company is to come as close to the blue prints as possible. They are working with a well-known problem, in a defined environment, with very few significant changes.

The “blue prints” in software development is commonly believed to be visual diagrams (think UML). But these diagrams are overviews and basically ideas on how to arrange responsibility and behaviour in a system. Given a diagram, no two programmers in the world will come up with the same program. From a model, the number of ways to create a program are infinite. These blue prints are everything but unambiguous and specific.

Let’s go back to design. What is design, really? Design is about finding the most suitable solution to a need from an infinite solution space. For every need, there are an infinite number of solutions. This can be illustrated with an example from food design: If you are hungry for fish, how many recipes do think exist that satisfy that need?

The goal of most design work is to come up with a recipe on how to perform a defined task as fast, correct and cheap as possible. If we design a factory for making cars, the factory operations plan is the design target. If we design a house, the blue print is the target. If we design food, the cooking recipe is the target. Everything we do from the point where we believe we have understood the problem up to the point where we have the recipe is design work, or support of design work. Everything we do after that point is construction, following an unambiguous and specified plan to construct something as quickly, correctly, and cheaply as possible.

If work until we have “the unambiguous and specified recipe” is design and the diagrams for software are not true recipes, then… then that must mean there is a lot of design work left when we start programming! It turns out this is exactly right. There is a heck of a lot design work to be done when programming. In fact, there is so much design left that many times moving to software invalidates assumptions we made when modelling the solution. Discoveries made when programming can change a design completely. Change is abundant. Work is undefined.

But what about the recipe? If design diagrams aren’t recipes then what is? Well, there is that one thing in software development that actually is unambiguous and specific. That thing is called source code. We could give the source code to somebody or something else and during construction they would always come up with the same result. And in fact, this is exactly what happens! We give the source code over to machines; compilers, linkers, and build servers, and let them have their way with it, thereby creating a working, executable program. This is a defined and repeatable process. The source code is specific enough even for machines to use it. This part of the process escapes many people since it’s quite technical, highly automated and very fast.

That is building software. In fact, that’s the word commonly used in terms like “build servers”.

Programming Is Design

This reasoning leads us to a better description:

Writing software = Designing programs.

The end result of the design is software, or source code:

The source code is our target recipe. The design lives in the source code.

Consequently:

Mainly, what a programmer does, is design.

It was fascinating to me how many things fell into place when I realised this (some years ago). Here are some conclusions:

  • We can’t design everything up front. We have to keep designing all the time during development.
  • Programmers are as much designers as architects and they need to talk to customers and users to understand what is needed. We can’t separate them without creating the wrong solution.
  • Design is mainly learned via observation and imitation of experts. Put programmers and architects on the same team to let the experienced designers teach the inexperienced.
  • We don’t have to make very elaborate diagrams. Sketching is fine, since it is most likely wrong anyway. Just enough to get us started is good.
  • Being an efficient graphic designer has nothing to do with how many pencils you have. The same can be said for programming and keyboards. The keyboard is not the bottleneck. Pair programming is a great design tool.
  • Being good at design is more about being skilful in your craft, experienced and willing to experiment with new things. It’s about how fast you learn what works and what doesn’t. Therefore, we should strive to maximise our learning.
  • We don’t have to monitor anyone. Let the designers get on with their business.
  • It is expected that we find errors in our experiments. This is a good thing since it will increase our learning.
  • Programmers that are allowed to meet their real customers and users will evolve a real sense of responsibility and care for both other people and their money.

Notice how every statement derived from the Big Misunderstanding is completely changed if we just consider programming to be design work.

A Reality Check

Now it’s up to you. If you believe that programming is like building a house, please perform a check against the reality that you inhabit. Do the requirements and designs from the start change? Do the programmers misunderstand the specs? Does programming really seem that simple? If so, why do the programmers look so concentrated when you walk by? Why doesn’t doubling the number of programmers double the progress of the project? Why do the developers feel they need to ask you questions even during the project? Why aren’t your customers and users happy?

21 Comments

  1. AMEN!!

  2. Social comments and analytics for this post…

    This post was mentioned on Twitter by jockeholm: @jelenavencl @danielbrolund @johlrogge OK! (Discl.: It’s long and not that angry. Didn’t have time to make it shorter.) http://bit.ly/avvyGZ

  3. (Standing up screaming like in British Parliament): Hear! Hear!

  4. Joakim Holm said,

    Daniel and Dan, thanks a lot for your enthusiastic cheers! :-)

  5. Rasmus Kaj said,

    Well put!

    Thank you!

  6. Viktor said,

    *clap… clap…*

  7. [...] more productive in the evenings then I am at the end of a working day. I also just finished reading Programming is all Design by Joakim Holman, Why Design Cannot be Billed by the Hour by Steve Zelle, Software on the Cheap by [...]

  8. Joakim Holm said,

    Thanks, Rasmus! Viktor, I can recognize a slow clap when I see one. :-)

  9. Excellent observation. Thank you for this post !!!

  10. Rob Cooper said,

    *clap… clap…* ;)

    Very, very well put!

  11. Joakim Holm said,

    Bjoern, Pierre, and Rob: Thanks guys! Glad y’all liked it.
    Pierre: An extra “thank you” for the endorsement in your blog.

  12. James Hicks said,

    Excellent article. One hurdle we can’t seem to jump is having developers communicate directly with end users. Some in my organization believe the business analyst/product owner is a sufficient user proxy 99.9% of the time.

    • Joakim Holm said,

      James: Thanks for your comment. I think your problem is very common. I think it’s only because of scale and tayloristic thinking that people think like that. Like if collaboration and looking to the whole would be a bad thing. Anybody who has ever worked in a small project, just one customer and one developer, know how effective that can be.

      One caution though: Most programmers are not trained in user communication. They don’t have the tools nor the skill set (like for example UX people), which can have negative effects. But in a cross-functional, complete team, why not start learning those skills now?

  13. harrychou said,

    In terms of communication, I have different thoughts. I guess designers have a little bit easier ways to communicate since they can use languages that users can relate to. Terms like emotions, warm, colors, … pop-culture actually helped to popularize some terms.
    Programmers either need to use other terms that is very abstract: flexible, maintainable, quality, … that are not easy to have a agreeable meanings; or they need to use geek languages such as database, SOA, cloud that will scare off people.
    I don’t know. I am not a designer, and this is only my guess.

    • Joakim Holm said,

      Hi harrychou! Thanks a lot for your input.

      Sure, I agree that programmers have a very difficult medium to explain to non-techies. But I don’t think we need to much. We need first and foremost to communicate with other programmers, mainly through our own designs. This communication has to take place over space and time. You can complement that channel with code walkthroughs, diagrams and documents. We may use a pattern language to communicate efficiently. Etcetera. But basically, the design has to speak for itself. Of course, the graphical and interaction design should be discussed with customers and users, but they’re are usually not that interested in the technical insides. Nor should they have to.

      In my article I am trying to guide the reader to my conclusion that programming has so many traits similar to more regular forms of design that we should simply view programming as mainly another form of design, which I think would have many benefits to programming as a profession. Remember, not all designs are visual. A poem has certainly been designed, to name just one example of non-visual designs. Or this comment! Every time you try and connect a fluffy need to an appropriate solution out of an infinite number in a design space you are designing.

  14. harrychou said,

    Oh, and designer’s work in progress is more visible too.

  15. Liz Keogh said,

    Brilliantly done. Are you also reading “Metaphors we live by” or is this just coincidence?

    • Joakim Holm said,

      Thank you, Liz! No, I’m not, but it sounds really interesting. I’ll check it out. Thanks.

  16. Adam Killander said,

    Great article! Standing ovation! Really liked the part about the source code being our target recipe.


Comments are closed.

Follow

Get every new post delivered to your Inbox.

Join 906 other followers

%d bloggers like this: