Some confusion arises from the fact that programmers like to change their mind on what they want to be called. We used to be programmers, then we became Software Developers and lately we have started calling ourselves Software Engineers, all while the things we actually do didn’t change much. So what’s up with that and why is it stupid?

Maybe at some time in the late 2000s I started to notice that people who I considered to be programmers called themselves Software Developers. Initially, I really liked that: The term acknowledges that making software involves more things than just programming. You gather requirements, design architectures, document your decisions, survey technologies, configure and integrate programs, deploy them, conduct user studies, … That all might be true, but I suspect it’s not the real reason people wanted to be called that instead of the meagerly “programmer”. The latter just doesn’t have the same ring to it: It evokes images of neck-bearded basement-dwellers who avoid other people, and live on energy drinks and pizza-rolls. Nothing wrong with that of course, but lots of people didn’t want to be thought of that way. The task of programming was seen as somewhat trivial, partly due to a movement that gained traction in the 90s where people believed that you could build software by having management-types draw pretty diagrams, hit the “Generate Code” button and out comes the finished program. Not only did that not really work, but together with increasing efforts to outsource programming jobs to far-away countries with lower wages and fewer employee rights and -protections, it contributed to a decline of programming’s prestige. After all, if we expect to be able to automate this task in a couple of years, then it can’t be that intellectually challenging. Well, it is and we couldn’t, although we are now in the middle of trying that whole thing again, just with different technologies (more on that later).

Programming is for nerds, Software Development is for cool people. I struggle to respect a terminology change that is done for the wrong reasons. I have no desire for my job description to sound impressive and that’s probably what this was all about. Because, while it’s true that Software Development is a more all-encompassing term for a job that includes many different tasks, that argument isn’t really convincing when we apply it to other professions. A baker doesn’t spend all his time kneading dough or pulling pastries out the oven; he manages his supplies of flour and whatever else he is using, he sets up the shift plan for his employees, he tries out new recipes, he analyzes which pastries sold well, he calculates the size of the next batch and the next order to place, he negotiates with suppliers… and yet, he is still a baker. Not a Pastry Developer, but a proud baker. What I’m saying is that of course any occupation includes multiple individual tasks. They might even be rather different from each other, some tasks are in preparation of the central one, some are meta-task, but there is such a thing as a central task!

It’s the same way with Software Development: Yes, you might spend more time in meetings than actually writing code, but the latter is still your central task. You might call yourself a Software Developer but the deliverable that you have to produce in the end is a program, is code. You don’t give your customer’s you whiteboard drawings, so whatever you do, there has to be some coding¹ involved. If you omit all the planning and coordination work that usually goes into making software, and you only program, the resulting program might not be great or might not do exactly what was asked, but at least you have one. You will have something that runs. If you exclusively plan and design and architect, but you never actually write code, then you won’t have anything! The program that will eventually get executed is the deliverable, creating it is the central task. Just as baked bread is the deliverable for the baker and baking it is the central task—hence the title baker.

That doesn’t mean that meetings are dumb and writing documentation is for losers. No, of course not. There’s a type of programmer that hates meetings and who thinks every bit of communication is a waste of time, the whole project could be done in a week if they just let him code away uninterrupted. These programmers often underestimate the complexity of the problem and overestimate their own ability. They just don’t like talking to people so they come up with reasons why they shouldn’t have to. I generally don’t think we should let them get away with that, but what makes it a bit difficult is that sometimes they are right. Sometimes they are actually as good as they think they are and sometimes the problem is actually not that complicated. Recognizing when proper planning and design are in order and when it’s best to just let them cook is the task of a team leader and it’s not an easy one. (I’d conjecture that being competent at judging when it’s best to let the programmers do some programming and when some more planning is in order, can give a manager and his team a substantial competitive advantage).

On the other hand there are people working as Software Developers who don’t actually like to program all that much and that are not particularly good at it and they know it. But in order to cover up their lack of skills they work to create as much process and structural overhead as possible. Because sitting in meetings is easy, programming is hard². So they downplay the importance of writing code and inflate the one of doing other stuff. Part of this shifting of priorities is choosing different terminology: Software Developer emphasizes the process around the central task.

So that explains Software Development but what about Software Engineering. This is a more recent trend even though Wikipedia tells me that it goes back to Margaret Hamilton. And I guess that makes sense because I don’t object to the use of that term in the context of something like NASA. But most people don’t work at NASA.

While the term apparently has been around for some time, it hasn’t made its way into as many job titles as it did in the last couple of years. What I find interesting about that is that it in some sense constitutes a counter-movement to the vague and squishy Software Development to focus back on the technical aspects of our profession. I consider that to be an indication of maturity for our field that we no longer want to get away from the dirty reality of programming. But what does Software Engineering mean or what does it imply?

I will argue that neither is the discipline of programming correctly categorized as engineering nor should it be!

For that let’s consider a spectrum on which we can classify activities between engineering on the one side and art on the other. I’ll use engineering instead of science, because I’d say that science is about creating knowledge, while the disciplines we care about here mainly involve applying knowledge (to create something).

Engineering aims to build something. There is an end-product you want to create and that product serves some purpose, it’s a means to an end. The product of engineering will be used in one way or the other and has to exhibit certain functional characteristics. It’s quality can be measured empirically and objectively.

Art on the other produces something that is an end to itself. The artifact is under no obligation to be useful, ergonomic, functional even. It’s purpose is of a totally different kind, to communicate an idea, evoke an emotion, give food for thought, entertain, any or all of those things or something else entirely. Whether or not and to what extent you can judge a piece of art objectively is a question I won’t go into here, but I will say that any such judgement would at least be qualitative and not quantitative: You can measure the horse power of an engine and that is a relevant quantity with which to evaluate the it, but you don’t measure the number of notes in a song and take that to judge the song. When you ask yourself whether you like a painting, you don’t count the number of brushstrokes it took or the number of colors used or its weight. You don’t empirically measure art to judge its quality. Whereas any product of engineering will be evaluated on relevant metrics that are not subjective in nature. I’d say that there is overlap in the reality of making art and engineering: An appliance, a house, a computer, any engineering product will incidentally have artistic properties, and there’s some science and engineering going into making art as well. That doesn’t invalidate the idea of distinguishing between disciplines that produce an artifact that serves a purpose for which its fitness can be empirically measured, versus those that create a thing for its own sake.

The definition of art that I’ve established here is notably different from the one Knuth uses in his talk about “Computer Programming as an Art”³. Using his understanding of the word, which he derives from its use and more specifically its distinction from and relation to that of science, I would agree with him that programming is an art form (also generally a good idea to agree with Knuth). But while it’s certainly interesting to observe how, according to him, science and art used to mean almost the same thing in the middle ages, after which the two terms drifted in meaning during a millennium of continued scientific discovery, and art was increasingly taken to describe that which is beyond scientific understanding, I’d say this perspective overemphasizes the process and overlooks the product of scientific or artistic endeavor. Knuth mentions how sometimes the term art would be used to describe the application of knowledge whereas science concerns itself with the creation of it. Since I’m interested in distinguishing art not from science but from engineering, which I’d also classify as application of knowledge, I choose to stick to the interpretation of art that I presented earlier. For that which Knuth calls art, I want to introduce a different term that hopefully better captures how I feel about programming: Craft.

While engineering is more rigorous in its methods and results in products with quantifiable properties, and art is more concerned with creative expression and an appreciation for the beauty of a thing in itself, a craft lies somewhere in the middle of that spectrum. We immediately associate that term with “traditional” crafts like carpentry. Building a stool or a table is an engineering project insofar as the product shall serve some purpose, it doesn’t exist to evoke some idea, but to use it. In order to create something that is adequate for its later use, certain rules have to be followed and engineering practices have to be applied in order to build a stool that doesn’t collapse under the weight of an average person. But surely a carpenter doesn’t need to work with the same rigour as the civil engineer tasked with building a bridge. An understanding of the sturdiness of wood from different trees is probably required as might be knowledge of the pros and cons of varying kinds of glue. But a carpenter won’t run simulations to verify the safety of his creation under extreme weather conditions, and he won’t build in an airbag under the seat in case the chair unexpectedly breaks down. He also doesn’t need a team of architects and designers and project managers to actually build it.

Much like the artist, the carpenter can oversee the whole project of building a piece of furniture by himself, from choosing the materials, maintaining the tools, to designing the piece, shaping its parts and assembling it into a whole. The final product is evaluated by others not only on its measurable quantities but also its immeasurable qualities: People rarely buy a chair for the exact amount of weight it can support or its aerodynamic properties, but for how comfortable it is and whether its look is to your taste. You don’t put it in a wind-tunnel, you sit on it.

This notion of craft is closely related to hand-crafted (or handmade) and it all sounds mildly romantic, but that’s on purpose. Maybe I’m conflating different categories here: I understand engineering to be more of an industrial undertaking and arts and craft to be… artisanal. The craft is the pre-industrial or maybe even de-industrialized engineering. It’s manufactured in the strict meaning of the word. And so is code.

Programming is seldom a purely artistic endeavor, as most of the time we want to build some piece of software that exists not as a end to itself but as a tool for people to use. Most programs serve as a means to an end and their effectiveness and efficiency at doing so can be evaluated. Some of the metrics we use are more empiric while others are harder to quantify, but we usually believe that there is some objective measure of quality. But that is not to say that those measures are the only ones we regard as important. We programmers sometimes care more about how the sausage is made than what actually comes out the other end. Terms like clean and elegant are part of our everyday vocabulary when talking about code. And while we can pretty well measure the time it takes for a program to transform its input data to some form of meaningful output, the aesthetics of code are much more in the eye of the beholder. That’s why I would place programming somewhere along the middle of the engineering-artistry spectrum. The relationship we programmers have with the code we write can be as intimate as the one between a craftsman and his work. The artifacts we create serve a function and have to prove their fitness for some purpose, but they can also exhibit immeasurable beauty.

The methods we use to achieve that, to create useful programs, often claim to be rigorous and scientific and solid, but they’re not. Much of the “knowledge” about how to best architect software programs, while being disguised as objective insight, are just some guy’s opinion. That doesn’t mean that there is no research going on into how to better build software and build better software, whether that pertains to the ease with which software is written, the speed at which it runs, or the assurances we can give as to its correctness. But findings from research only slowly trickle into to real-world practice and how to adequately apply them is not always obvious.

For a long time researchers have been occupied with finding methods to prove the correctness of programs although I have to admit that I’m not at all up to date with the state of the art in model-checking and formal verification. But I do think we saw an increasing interest from our community in functional programming languages or at least parts of the functional paradigm being incorporated into languages, which seems to signify a new focus on correctness coinciding with programmers rebranding themselves as Software Engineers. So it seems at least consistent for people who like to be considered engineers to adopt more rigorous approaches, in which programming is no longer thought of as a craft. It should be said that this is not the first time these ideas have been part of the mainstream: The techniques used in the 90s were different, with less focus on the code and more on other descriptions of what a program should do. In that paradigm, you would rigorously specify a program in terms of UML diagrams and then generate the obvious code from that (I mentioned that earlier). It turned out that just because you use a different notation doesn’t mean you won’t make mistakes, and any high-level description of what a program should do either ignores some aspects of the program (which would then have to be baked into the code generation stage), or you make the formalism so complex that it is no longer high-level and instead isomorphic to the code being generated. After it was clear how that wasn’t going to “solve” software, the community moved on to greener pastures. The web was new and chaotic, and, tempted by the allure of big dotcom-money we abandoned correctness in favor of time-to-market (maybe we abandoned a bunch of other important qualities too). The new wave of interest in correctness—or, to use a term I find more problematic: safety—that we now see, is more focused on the code itself, which is a good thing. As I said above, the code is the deliverable so it should remain front-and-center in our processes and methodologies.

We seem to be seeing a shift towards programming languages and ecosystems with most robust memory models, complex type systems or even a focus on pure functions, and forgoing these affordances is looked down upon: Using plain JavaScript instead of TypeScript is considered unorthodox, using C or C++ is heretical! This might be what the Software Engineers want, but it changes the character of our discipline. Earlier I explained why I consider programming to be a craft, now let me say that I prefer that we keep it that way.

Adoption of more rigorous methodologies might be motivated by a desire to build more reliable software (which is commendable) or a latent inferiority-complex of some practitioners of the craft. Or maybe we have discovered some old traditions to be based on lies and we need to fill the void that was left behind after ripping out the previous ideology. If it really is about an honest interest in improving the quality of software, then we seriously need to consider which techniques are actually effective helping us in that regard, and with what kinds of trade offs they come.

There are two specific points I want to point out with respects to the dangers of the shifting sensibilities in community. The first concerns the idea of safety and its exclusionary agenda. I alluded to me finding that word problematic and this is why. It implicitly claims a moral high ground while remaining fairly vague. Of course the word expresses something, but it is not specific and combined with its completely positive connotation, it lends itself to abuse. Words like this can be wielded to justify all kinds of different things with the superpower to discredit anyone arguing against it—after all: who would want to speak up against such a good thing as safety?! Ideologically charged words can and will always be used to exercise some form of social or political power; the same way that totalitarian regimes have used vague but (each in their context) positively connoted words like patriotism or revolutionary to provide justification for whatever decision their leaders wanted to enforce. Nobody wanted to be unpatriotic or counter-revolutionary as those were the worst offences in their respective systems of believe. So nobody could speak up against policies or institutions that claimed to operate in the spirit of these values. Is it overly dramatic to make this comparison when talking about methods for building software? Maybe, but the dramatization clarifies the idea quite nicely. The appeal to safe programming practices is starting to get used to delegitimize different viewpoints and common practices of the craft.

An additional consequence of the misunderstood obsession with very specific notions of safety, and that is the second point, is to extend the rhetorical sanctioning of heretical opinions to the realm of legality. The EU is increasingly active with regards to the noble cause of protecting its citizens from harm in the digital world. Companies are now punished for gross offences when it comes to data privacy leaks and abuses, which seems like a good thing. We also see laws themselves undermine citizens’ privacy (also justified by a vague notion of safety) and demand social networks censor content in order for people to not get their soft little feelings hurt. The fact that government legislation often misses the mark by misunderstanding the realities in an area of work is nothing new and under the influence of certain safety propaganda I immensely fear that ill-advised lawmakers could get the idea to hold anyone who publishes software liable for anything the software does whether by working correctly or by misbehaving unintentionally. This does not seem totally implausible to me and if implemented would completely annihilate the world of Free and Open Source Software that has created so much value for our civilization. Someone will show the politicians how all Free Software licenses contain language like this (from the MIT license):

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

That might just make the lawyerly brain of some industrious regulator explode. “No warranty? No liability? That cannot be!”. In the name of safety, laws might get passed that hold anyone creating or publishing software liable for anything the software does, even if the software was provided free of charge and with a warranty disclaimer like the one above. And that will be the end of Free Software. Because who can afford to provide a warranty and to take on the responsibility?! Independent developers sure can’t, only the big corporations will begrudgingly comply with such regulation. They won’t be happy about it, but at least they got the pesky little open source projects off their back that endanger their monopolies. Lawmakers will actually smash two birds with one stone by doing so, because Free and Open Source Software will be increasingly considered to be for nefarious purposes only: It can be used to communicate securely using encrypted channels (very dangerous, only criminals do that) and distribute information freely without government censorship (also very dangerous, only subversive elements would want access to uncensored information). Nonfree software could do those things as well, but companies are easier to control than decentralized groups of civilians. Again, maybe I’m being a slightly overdramatic here, but I wouldn’t consider this to need saying if I wasn’t afraid of it.

The safety people in the programming community will say that that’s just how the cookie crumbles, because they are employed by those companies and earning big money as Software Engineers. The collateral damage will be the one who cares about users’ rights and software freedom: The little man, the hobby programmer, the user-developer, the has-been Hacker, the Code Craftsman.