Programming Is All Design

There is one, Big Misunderstanding underlying a lot of problems in software development today. This misunderstanding is the reason why programmers are treated as if they were orcs, why business guys have such a hard time collaborating with IT projects, and why outsourcing of software development is generally considered to be a truly great idea. Here it is:

The Big Misunderstanding

Writing software = Building programs.

Most people, even in IT, believe this analogy to be right and good. If you do, then the following statements should also seem basically correct, since they are all based on that view:

• Customers don’t have to talk to everybody. They should confine their communication to the smart guys. We’ll call them architects. These are the guys customers talk to during project inception to get the details flushed out. After this, the architects take over and handle matters.
• We should focus on getting the blue prints (the plan, the model, the spec) correct from the start. Let the architects draw diagrams and write specifications for the programmers to implement later.
• Building programs is a conversion process from drawings and specifications to code. A programmer’s job is to perform this conversion as accurately as possible.
• Programming is mainly manual labour. The more programmers and keyboards we have, the quicker the implementation will be.
• We should monitor the programmers to make sure they don’t deviate too much from the specification we have painstakingly created. They should simply build the programs accordingly and all will be fine.
• If a programmer should find an error (not likely), she would have to prove it to be an error before we can act on it. Changes cost a lot of money, something a programmer probably does not see or care about.

Maybe I’m overdoing it, but I hope that some or even all of these statements sound familiar. If they do, it’s all because of the Big Misunderstanding. They’re all connected and they’re all wrong. Fundamentally wrong!

Where Did These Ideas Come From?

Software development is often compared to the construction industry. Precisely why we admire that field so much is a little hazy. My theory is that it’s a version of something coders call Feature Envy. We want so desperately software development to be considered a serious field, like civil engineering, and what could be more big, manly and serious than being (like) a civil engineer, blowing up stuff and erecting towers. So we ended up with “architects” that produce schematic drawings of software and construction workers (programmers) “building” the software exactly from specification.

But what if the assumption that software development is similar to civil engineering is wrong? What if the implications of engineering lead us to a warped picture of what software development really is? What if this is misleading and generates problems instead?

To determine the truth we need to re-check our assumptions of the nature of software.

Design Targets

The artifacts delivered by a building architect are, more or less, unambiguous and specific. Yes, there is room for the occasional discussion, but the main job of the building company is to come as close to the blue prints as possible. They are working with a well-known problem, in a defined environment, with very few significant changes.

The “blue prints” in software development is commonly believed to be visual diagrams (think UML). But these diagrams are overviews and basically ideas on how to arrange responsibility and behaviour in a system. Given a diagram, no two programmers in the world will come up with the same program. From a model, the number of ways to create a program are infinite. These blue prints are everything but unambiguous and specific.

Let’s go back to design. What is design, really? Design is about finding the most suitable solution to a need from an infinite solution space. For every need, there are an infinite number of solutions. This can be illustrated with an example from food design: If you are hungry for fish, how many recipes do think exist that satisfy that need?

The goal of most design work is to come up with a recipe on how to perform a defined task as fast, correct and cheap as possible. If we design a factory for making cars, the factory operations plan is the design target. If we design a house, the blue print is the target. If we design food, the cooking recipe is the target. Everything we do from the point where we believe we have understood the problem up to the point where we have the recipe is design work, or support of design work. Everything we do after that point is construction, following an unambiguous and specified plan to construct something as quickly, correctly, and cheaply as possible.

If work until we have “the unambiguous and specified recipe” is design and the diagrams for software are not true recipes, then… then that must mean there is a lot of design work left when we start programming! It turns out this is exactly right. There is a heck of a lot design work to be done when programming. In fact, there is so much design left that many times moving to software invalidates assumptions we made when modelling the solution. Discoveries made when programming can change a design completely. Change is abundant. Work is undefined.

But what about the recipe? If design diagrams aren’t recipes then what is? Well, there is that one thing in software development that actually is unambiguous and specific. That thing is called source code. We could give the source code to somebody or something else and during construction they would always come up with the same result. And in fact, this is exactly what happens! We give the source code over to machines; compilers, linkers, and build servers, and let them have their way with it, thereby creating a working, executable program. This is a defined and repeatable process. The source code is specific enough even for machines to use it. This part of the process escapes many people since it’s quite technical, highly automated and very fast.

That is building software. In fact, that’s the word commonly used in terms like “build servers”.

Programming Is Design

This reasoning leads us to a better description:

Writing software = Designing programs.

The end result of the design is software, or source code:

The source code is our target recipe. The design lives in the source code.

Consequently:

Mainly, what a programmer does, is design.

It was fascinating to me how many things fell into place when I realised this (some years ago). Here are some conclusions:

• We can’t design everything up front. We have to keep designing all the time during development.
• Programmers are as much designers as architects and they need to talk to customers and users to understand what is needed. We can’t separate them without creating the wrong solution.
• Design is mainly learned via observation and imitation of experts. Put programmers and architects on the same team to let the experienced designers teach the inexperienced.
• We don’t have to make very elaborate diagrams. Sketching is fine, since it is most likely wrong anyway. Just enough to get us started is good.
• Being an efficient graphic designer has nothing to do with how many pencils you have. The same can be said for programming and keyboards. The keyboard is not the bottleneck. Pair programming is a great design tool.
• Being good at design is more about being skilful in your craft, experienced and willing to experiment with new things. It’s about how fast you learn what works and what doesn’t. Therefore, we should strive to maximise our learning.
• We don’t have to monitor anyone. Let the designers get on with their business.
• It is expected that we find errors in our experiments. This is a good thing since it will increase our learning.
• Programmers that are allowed to meet their real customers and users will evolve a real sense of responsibility and care for both other people and their money.

Notice how every statement derived from the Big Misunderstanding is completely changed if we just consider programming to be design work.

A Reality Check

Now it’s up to you. If you believe that programming is like building a house, please perform a check against the reality that you inhabit. Do the requirements and designs from the start change? Do the programmers misunderstand the specs? Does programming really seem that simple? If so, why do the programmers look so concentrated when you walk by? Why doesn’t doubling the number of programmers double the progress of the project? Why do the developers feel they need to ask you questions even during the project? Why aren’t your customers and users happy?