What sort of developer are you?

The profession, trade or craft of software development is an exciting one to be in, and part of that is precisely because there is no clear understanding whether it is in fact a profession, trade or craft, or even whether or not software development is a thing in itself, distinct from say programming, computer science, or software engineering. There are a lot of strongly held and expressed opinions out there about it, whatever it is, and I too have some things that I absolutely agree with, and many things that I believe are mere matters of taste, highly context dependent, or I am simply undecided about. As unhelpful as it often is to rely on labeling things, especially people, the opinions that people have are generally shared by some and opposed by others, and this leads to being able to demarcate communities that have in some cases fairly long and rich histories. The interplay of all these ideas provides for an exciting and dynamic landscape for those of us involved in software development, and we are pretty lucky that we can each form our own visions of how we think the future of the profession should look, and usually have the fortune to find positions that suit those visions perfectly.

First of all, in order to put things into perspective, lets have a look at the general software development landscape. A lot of disagreements and misunderstandings between the various software development communities arise because people have very different backgrounds, and do very different things, which means what one developer finds valuable or inspiring might not necessarily be helpful to another developer doing different types of work. I see the distribution of the number of developers engaged in work requiring different skill levels as being distributed under a log-normal type of graph, with the bulk of software developers employed in fairly undemanding work. An example would be shifting data between a database and a user interface with some reasonably simple and well understood business logic in between (or there is at least a domain expert available with knowledge that is fairly close in form to what the programmer is expected to implement), or perhaps plugging some libraries together. In the old days there used to be a division between applications and systems programmers but I am not so sure this is that relevant anymore. As you move right along the graph you have a fewer number of more highly skilled developers, perhaps because they have specialized skills, or they need to be really good generalists, or they are inventing whole new things themselves. Over here the developers either have to be the domain specialists, either in a non-software domain (such as financial engineering, some branch of mathematics or other science, electronics, entertainment), or in a domain related to software development such as advanced algorithms and data structures, AI, large distributed systems, in depth hardware knowledge, or even in some non-programming aspect of software development such as testing, technical writing, coaching, requirements engineering or management.

So the stereotypes at each extreme are at one end, which I am going to call the "low end" the 9 to 5, enterprise software developer (started off doing maintenance but got promoted after five years because he was in maintenance the longest, most people quit after two years), who takes specifications from an analyst or "designer" (possibly a new record that the DBA has already produced the schema for needs a user interface where the user can edit and save), implements them as software (by cutting and pasting the code for a similar record and changing the fields), and then passes the software along to the QA stage. He is likely doing this in COBOL, Java or C#. Perhaps C++. He might not know or understand how this bit of software fits into the larger product, nor who the customer is, so he doesn’t feel particularly responsible if things go wrong. Usually when QA finds a bug it was the analysts or designer’s fault. He has heard that the maintenance team criticizes his code for being difficult to maintain, but meh, those guys always complain. He probably didn’t learn programming as a child, and doesn’t program at home. He doesn’t read programming books, blogs, belong to mailing lists or go to user groups. He has been on stack overflow when googling some questions but doesn’t have an account. He did hear of XP once, and was amused by it, and hoped management wouldn’t introduce it at his company. He has not heard of software craftsmanship.

At the other end, the "high" end we might have someone who learned to program as a child, managed to get a computer science degree despite being bored by most of it, and distracted with his own projects, started off enterprise programming before he discovered how boring it was, and started on his own as an independent consultant, specializing in the open source project he has been contributing to for a few years, written in a dynamic and/or functional language. In his own time he is active on twitter, blogs and mailing lists, speaks at the local user group, and even gave a talk with another developer at a national conference. Along the way he got his masters degree. It doesn’t matter what in, could have been something computery, might have been something else. He was skeptical how relevant it would be, but he made sure to pick a decent project that would give him a sound theoretical basis as well as practical experience in a topic that he was interested in, and has discovered some demand for. Recently, he has been thinking about the relationship he has with his clients (many household names). Some of them just have stupid ideas, and the few with good ideas make a lot of money for doing not much else than contracting him to do the actual work. So in his spare time he has been working on his own web based business, filling a financially lucrative niche that has yet to be automated with a decent user friendly set of tools. He has heard of XP, and was even using some of the practices sometimes, when they made sense, before he heard of XP. He has friends (in the enterprise sector) that are right into it though can’t see it really suiting what he does (sustainable pace? why not just find a job you enjoy as much as the programming you do at home to all hours of the morning?). He has heard of software craftsmanship too, but isn’t quite sure what it is.

Please keep in mind that these stereotypes are just that, examples to help form an illustration, I could have probably written a handful of descriptions, all completely different from each other, especially to describe the more skilled end of the spectrum. I would say advanced computer scientists in academia are possible alternative examples of software developers at the "high end" depending what they do of course, and how you define software development, as are the inventors of good tools and frameworks, and people with the energy and vision to just create great products. All those people you read about in Beautiful Code, and Coders at Work etc I would place at the high end.

So where am I? Well I am probably in the middle somewhere. Doing what I can to move towards the right, but my theoretical learning and playing around with toys as home far outweighs the real practical experience I have been able to get, so to get anywhere I am going to have to redress that balance, stop learning and start doing or something. Also if I map myself to the apprentice, journeyman, master, or shu ha ri models, I am probably, if you aggregate a basket of software development skills into a single abstract "software development generalism" skill, then I am probably in the middle of those too, at the journeyman or ha level.

So one of the main debates in software development is the issue of whether it is an engineering discipline or not. I think it sort of is, especially at the low end. Now Software Engineering for me was the stream in Computer Science that focused on practical matters. I remember as part of the software engineering we covered topics like coupling and cohesion (which was then as close as anyone came to things like the SOLID principles), source control (back when few in industry were doing it), and standards software development life cycle models, such as the waterfall model (oh well, two out of three ain’t bad). Initially I was confused when talking to "believers in" software craftsmanship as they seemed to criticize both the lack of practical stuff in computer science, as well as software engineering. Isn’t software engineering the part of computer science that address what you were missing. But it came down to a misunderstanding. Software engineering is actually now fairly well defined in its own body of knowledge, the SWEBOK. Now I had a look at the SWEBOK, and it is not all bad, but I definitely think it takes the engineering metaphor further than the nature of software development lends itself too. My main beef with the SWEBOK, and other engineering-like approaches to developing software, is they think design and implementation are two separate things. One of my most strongly held opinions is my agreement with Jack Reeves’ Code as Design essay. Most engineering inspired models for software development think software is like a building. The design phase produces blue prints, or some sort of design specifications for software, then the building, or software is constructed from those designs, and by constructing software they mean producing source code from the specifications. Now I agree that software can be designed and built, but I map them to the building metaphor differently. For me the source code is the blue prints. Construction is the very cheap, automated process of compiling the source, if relevant, and packing it up for deployment. Because this construction part is so cheap, compared to constructing builds, it makes other aspects of engineering less relevant. We just don’t need to go through a series of formal documentation handover phases first, and in fact doing so is harmful. When constructing a building, you need to make sure the design is right first, because once built changes are expensive or even impossible. With software you just make a change and restart the automated, compile, test, deploy process. If you can do that while sitting with the customer, you can probably do it dozens of times (if you don’t get it right before that) in the time it takes to start again with the whole formal pass-the-document procedure.

One thing I am clear on, as a developer I know things work better when working closely with the customer, delivering small chunks of functionality, in short feedback cycles, than it does according to more elaborate engineering like processes. Some people say more serious, larger scale operations require the extra formality and discipline that a software engineering approach provides, but my opinion would be to try and scale things down first. I heard the SWEBOK was up for a revision to take into account some of the changes the software community have been making to streamline away from such heavyweight models but a look at the website doesn’t provide much indication that any such "radical" revision is underway.

My main reason for not liking the term Software Engineer to describe myself is because I don’t have an engineering degree, am not certified or licensed as an engineer, and do not belong to any engineering professional societies. Also I don’t believe that for most of the work (i.e. the trivial shuffling of data around according to well known and understood patterns) that software developers do, the extra overhead of licensing and certification is justified, however I do concede that for more "mission critical" operations, or in situations where the customer is willing to pay for extra assurance, some form of certification is needed to sort the wheat from the chaff.

A reformed and updated concept of Software Engineering could be very useful for the low end, where developers just lack the skill to deliver a quality product from concept to cash. But the SWEBOK version of software engineering is not it. A cross functional team, some systems thinking, engaged leadership, scaling things down to a manageable size, fighting against complexity, and a smattering of XP practices will always do a better job of optimizing things at the low end than the SWEBOK.

So is software development a craft? The book Software Craftsmanship was written as a rebuttal against the engineering model for software development, and suggested instead a model based on the apprentice, journeyman, master levels of old fashioned craft guilds, as well as, at least on the part of a master, long term relationships with customers and code (high quality maintainable code, rather than throwaway code). The community seems to mostly address the needs of the low end, how an apprentice becomes a journeyman, encouraging poor enterprise developers to approach a level that most of us would consider acceptable, the focus is very similar to XP, with some of the SOLID and other clean code principles thrown in. There is also a "community of practice" component which I find most interesting, and am glad to be attending the Software Craftsmanship event in Bletchley Park this year. They way I approach my work, I definitely feel more like a craftsman than a engineer, so am happy to be considered a software craftsman, but I don’t know if the crafts approach to work is enough of a defining characteristic for me to actually call myself a software craftsman rather than a software developer. I guess it is just one "tool" in the box or more appropriately community I can draw from. Two things concern me though. There is little discussion about what a master is, apart from the idea of approaching mastery through practice. For me it is important though. As a journeyman my focus is on learning, and spending just a few years at each employer, before moving on to the next. At some point, to completely fulfill the craftsmanship vision I will need to switch that focus from learning to teaching, and from short term involvement with code and customers, to a long term one. It is still unclear how that point will be determined. The community seems happy thinking there are no masters, and we are all lifelong journeymen approaching mastery, but at some point, someone has to take a long term responsibility for the customer and their code. The other thing that concerns me, is that for many people who I think are high level developers, or at least people on forums like proggit and hacker news that I think I can learn from, craftsmanship doesn’t seem to offer them anything, they see it as philosophical, or religious propaganda, consultant-speak rather than actual coders being behind it, re-branding old ideas, marketing, etc. Just search reddit for software craftsmanship and look at the comments. Agile gets a similar reaction, and they are partially right. Where do Knuth, Torvalds and Norvig stand on the whole agile/craftsmanship thing? Most of the people behind it seem to be more well known for the books they sell, and the training they offer than the actual code they write.

I maintain both agile and craftsmanship are great for most developers at some points in their careers. Ideas from both will shift you slightly towards the higher end. I see the sweet spot for both as being in the middle of the low end, or perhaps the transition between apprentice and journeyman. I would say you need to understand both to be considered a decent developer writing say business software, but once the XP practices, and SOLID principles etc have become part of your habit, there isn’t much more to offer. If you are already working in the higher end, then ideas from agile and craftsmanship are either already part of the way you work, or simply not relevant.

What about computer science and software development? They are different right? But is the fact they are different mean they complement each other, or does the difference represent an undesirable failure of one to bow to the needs of the other? I see a lot of anti-academic vibes coming out of the software development community. Some of them have legitimate points, and advocate reform of the curriculum, and more cooperation between industry and academia. I would agree with that, but the vast majority is simply criticizing computer science departments for wasting time on useless theory and not being a trade school and churning out experienced developers ready to serve the needs of industry. I disagree with that, but see where they are coming from. First of all they are separate disciplines. They have a bit to do with each other. Computer scientists often need programming ability to a certain extent, and some software developers, especially at the high end, need to draw upon their computer science background to enable them to solve problems in efficient ways. The degree of overlap varies a lot depending on what you are doing. At the low end, you just need to know your programming language, and your library, a handful of patterns, and the domain specialist tells you the technical details, and you are good to go. Perhaps throw in some XP practices, and SOLID principles to make yourself more effective, and that is pretty much it. But at some point, as the problems become more technical in nature, and require innovative solutions rather than say business problems that have existing solutions that need automating, then you will be expected to be domain specialist in algorithms and data structures. Imagine working for a chip testing company. The task is to simulate the behavior of a microchip under certain circumstances. So you ask the domain specialist how that should be done. He is a chip designer, he can tell you about the hardware, but it is all on you to work out how to do it in software. Compare that to enterprise software, where the domain specialist can usually give you a formula or validation criteria that are usually fairly trivial to implement in software. So once you are a journeyman, and are writing clean enterprise code, effectively with a selection of XP practices, you might want to move on to more advanced software development, and your computer science background will come in useful.

Not for everyone though. I said there are a lot of different ways the high end could look, there are a lot of things a great developer can do without computer science. They could be leveraging a great idea, or some other specialist domain knowledge, or maybe they are highly specialized in some technology for example.

So generally I disagree with the anti-academics, find it sad that an institution that has formed such a strong pillar of our heritage as software developers is getting such little respect, but I do understand why a lot of developers fail to see the relevance of computer science. Almost all enterprise coders, just shuffling data between a database and a UI don’t need it, and even many great high end programmers find ways to excel without it. I would say for most people though, wanting to leave the boring enterprise world, for something more interesting and demanding are going to be better off having a computer science background in their toolkit.

Another difference between developers is those who prefer customer contact vs those who would rather spend all of their time behind the keyboard. Once again my preference depends on the context. If the programming work is low end, then it might get too boring, so let me mix it up by exposing me to other parts of the value stream. Who knows, I might even be able to contribute more by offering my technical perspective at the point where the customer is deciding what he wants. On the other hand, at the high end, it might be hard to tear me away from the keyboard, although I can’t really imagine a situation where I would really prefer no customer contact over close customer contact, no matter how interesting the programming itself is. The same applies for contact with other developers. Give me a highly collaborative, busy, slightly noisy team room (ideally with developers, testers, analysts etc all together) over being shut up alone in an office for days on end, but once again it depends on the nature of the work. Some tasks just don’t lend themselves to collaboration. I tend to approach finding and fixing bugs for example in a way that doesn’t lend itself to close collaboration. I should blog about that sometime. It probably has as much to do with personality and they way we approach our work as it does the nature of the work itself. I would say both types of developer are just as good, and a team is probably stronger for having both types on board.

Apes vs Vulcans: One interesting blog post I read a while ago talked about apes and Vulcans. Rather than have me do a poor summary, I suggest you read it yourself. Generally I see Apes being more relevant than Vulcans at the low end, but at the high end I think both can be present. I think it is possible to be a great high end ape, or a great high end Vulcan, and the community needs both. I think I can adapt to either role, but because I have spent too much time doing the sort of work that has only limited technical scope for improvement, or at least working at places where the pain points are cultural rather than technical, I have been being much more of an ape recently. I suspect this dichotomy is part of the reason why I see such a negative reaction towards Agile and software craftsmanship on reddit and Hacker News. Agile and craftsmanship are pretty apey, while the proggit crowd are mostly Vulcan. Hmm Hacker News actually seem more Ape than Vulcan, but they are high end apes on Hacker News, whereas Agile and Craftsmanship, as I already mentioned above, seem more relevant to low-end developers.

A tale of two programmers – Steve and Chris: Here it is clear. I am definitely a Steve. I do the important/difficult/risky bit first, to prove it can be done, then I might make it into a usable program, but then I find all the finishing touches too boring to worry about. I would gladly allow a Chris to take my core functionality and add a user interface, or take my architecture and do all the copy and pasting required to make it complete. I can’t be a Chris, because I don’t like working within the constraints of someone else’s decisions, I am likely to change more that I should, rewrite, or do a sloppy job, because I was given such a poor foundation to build on anyway. I definitely need to improve my ability to finish, and perhaps be more tolerant of the decisions that previous developers have made in code I am expected to work in.

So what sort of developer are you? Is software development a trade, profession or craft and why? What movements have helped you along the way? What ideas are you looking at to help you through the next step along the journey? Are you some interesting type of high end developer that I haven’t mentioned? Perhaps you could describe your job, or should I say life, as a developer and maybe inspire others wanting to move into the high end. How do you feel about computer science? Useful tools, or an outdated relic? Do you like contact with customers and other developers, or would you rather go it alone? Are you an ape or a Vulcan? A Steve or a Chris? How do you model the landscape of software development, and where do you fit into it? What sort of developer are you?