The weekend before last, I went to the second .NET Open Space in Leipzig. Since the last one I have actually attended two other open spaces, a Software Architecture one in Düsseldorf, and another .NET Open Space in Ulm, which were great, but I never got around to blogging about them.

On the first day the sessions I attended were:

• Requirements management
• DDD
• Making software development more attractive
• Scrum
• CQS and event sourcing

On day two I attended:

• Technology independent architecture
• A coding dojo (as audience member rather than hands-on participant)
• REST
• Developer training

The making software development more attractive one was interesting. I still maintain we should be raising the bar rather than lowering it, and encouraging fewer of the right people rather than simply more people. Quality over quantity right? This was an issue in the late 90s when computer science departments started dropping Scheme for Java, concentrating less on computer science theory and more on churning out applets and web pages for the “WWW boom”. When the bubble burst, a lot of hipsters who couldn’t code had to find new careers. Read CACM and you will see that universities still want those days back, when COSC 101 was one of the bigger must-do classes at uni, but I think industry seems a lot more concerned with the quality of developers than the quantity, and university computer science departments, as in other subjects should be concentrating on the theoretical foundations of computer science,  software engineering principles, and further exploration of ideas that are still considered academic and have not yet reached mainstream industry acceptance. The university should not try and replicate the workplace and churn out workers. They should not be vocational schools. You only have a few years at uni, but you have a whole career in the workplace to pick up more practical experience on larger, longer projects with real customers etc. In short, don’t worry about making it more attractive by broadening its appeal (i.e. lowering the bar). If you do want to make it more attractive, make it more professional. The right people will always consider software development a career, but until software development becomes a first class chartered engineering discipline, or something with equivalent professionalism, then the brightest minds will probably be more attracted towards more “nobler” vocations such as (real) engineering, accountancy, law or medicine.

The making software development more attractive track was large enough that it should have been a fishbowl I feel. The same voices dominated that particular discussion. The only way anyone got a chance to talk was by interrupting someone else, and everyone who spoke felt compelled to talk really quickly so they could get their point across before they were interrupted.

CQS and Event Sourcing was a topic suggested by me. (Yes I should have specified CQRS) I had hoped that someone had practical experience implementing these patterns in .NET and was hoping to hear what tools they used, what issues they had etc. Nobody had such practical experience with it, but a small group of us had a nice discussion about it anyway.

The coding dojo was interesting. It was based on Kata Potter. I was initially too scared to participate, assuming that much smarter people than me would bang out a solution in 5 minutes. But after watching it for a bit, I almost felt like jumping up and taking over and at least getting them started. They were too busy thinking about the tricky cases, and what sort of data structures they needed for the internal algorithm, and didn’t really get anywhere in the hour I watched them. Each person started writing out a test, which was good, but not really clearly worded to specify what behaviour is expected. As a result it was not always clear how many things the test was supposed to be testing. When it came to implementing the functionality to make the test pass, they kept on going back and forward changing how the data should be put in to the system, and getting side tracked in discussion about details. I am pretty sure I would have done a better job, mostly by naming my tests better, so it is clear what is being tested, and implementing just the minimum functionality to make the test pass without worrying too much about whether the data structures are ideal for the tricky cases. I would probably start off with a Book and an Order class, and if it turns out to be the wrong approach I would just change it later once I have a deeper understanding of things. But I can’t really make any such claims until I try it for myself, which I intend to do, sometime. I might even try it with Haskell or F#.

REST was interesting. It was mostly focused on how it can be done with WCF and ASP.NET MVC, rather than REST principles themselves.

The discussion about developer training was interesting too, although I was pretty tired by that stage. I assumed it was mostly about how a developer can improve their lives and career, as they specifically mentioned during planning about discussing options or possible future prospects for older developers. It seemed to focus mainly on how companies can improve their businesses by sending their developers off to training, which to me is a totally different thing to developers setting their own goals and shaping their own careers by choosing their own training options. I would have preferred more discussion on the latter.

In the end however, I think I am starting to lose interest or motivation in community participation. Probably because most of the “community” I have been associating with is fairly .NET focused, and discussing fairly new tools and techniques, whereas at work things seem to moving more towards the old, legacy, C++ maintenance work. It is hard to contribute to a discussion on, say DDD (which is in no way .NET specific at all) when you only have theoretical understanding of it, and no current practical experience with it in the workplace. Unfortunately there was no track on maintenance of old legacy C++ software, but even if there was I feel I would have more questions than answers. For a similar reason I am less motivated to attend (.NET) user group meetings, it feels a little disheartening to see all these cool new toys and know there is no place for them at work, and it feels like a poor use of my time to start learning about things I am unlikely to get a chance to apply practically to the delivery of customer value. Perhaps if there were something with a less specific focus, such as a Software Craftsmanship user group, I might find that a little more inspiring and motivating, and be able to apply it directly to my job in software maintenance.

Anyway, thanks a lot to the organisers of the .NET Open Space 2009!

I have been at my current employer for six months now, and it’s clear that it’s not a software development job at all. It’s a software maintenance job. I have noticed a lot of differences. There seems to be a lot less opportunities for creativity, or innovative ideas, as everything has already been decided. It reminds me a lot of the assignments we would set for the first year computer science students at uni. We provided the skeleton of the program, a bit more than method stubs, and allowed the students to fill in the gaps. In fact, in my job, even the gaps are filled in, and I just make a few changes here and there. Another difference is in the flow of delivering value. I am used to hammering out the tests, hammering out the code to make them pass, running all tests and checking everything in, and it all feels like delivering value. I hardly ever used the debugger, only in situations where something really weird was going on. Now, I basically start off in the debugger, spend a few days stepping through code, writing down notes, keeping a note of what methods and classes really do (as opposed to what they are called, which is more related to what they might have done some years ago), and trying to link things back to domain concepts if possible (although that’s not often so useful, another difference between development and maintenance seems to be that technical aspects of programming e.g. classes, xml, bytes, etc become the actual domain concepts rather than orders, customers, products etc). A lot of time is spent waiting for some expert, usually one of the other maintainers that developed the software, to explain something, and at the end of the week the change is perhaps less than 20 lines. I felt like something was wrong, as if such a small part of my time was spent truly delivering value. I was comparing apples to oranges again. Something that would have taken half a day in a development context, seems to take roughly 10 times as long in a maintenance context. But this is nothing new. Everyone knows that maintenance is more expensive than development, that is why software is often developed as projects, with fixed time-spans for development and maintenance before it is retired. But sometimes the business reality is that is in fact cheaper to continually develop something than start again from scratch. I have also heard it said that replacing a legacy system can also be expensive. So there is clearly a business case for extended maintenance of mature software products.

I wasn’t too interested at first, I wanted to be a craftsman, a creator of new software, a developer. I wanted to play a part in the high level architecture, and be involved in the decision making from a project’s very beginnings. However, software maintenance does have something to offer. It seems that there is huge money in it for a start. Customers see it as a lower risk, quicker turnaround alternative to developing a new solution. The fact that it is so expensive suggests to me that there are still some unsolved problems. Considering myself as a developer, yet engaging day-to-day in maintenance felt frustrating, so I am accepting reality, and am now considering myself a software maintainer.

However I am still a NNPP after six months, and that is unsatisfying. I take up more of the original developer’s time getting him to explain something than if he were to simply perform the maintenance himself. It is a classic case of Brook’s “Mythical Man Month”. An article in the BCS mentions “Whatever the maintenance required, it must be fully understood by the maintainer, including the impact of the maintenance in terms of cost and effort to implement.” Everything I read says it makes more sense to have the original developers perform the maintenance, and adding more developers simply makes it slower, and that if a company really feels that they have so much to do that they need new developers, then that new functionality, should be implemented by the new developers in a new project. However, the people who make decisions obviously have access to information that confirms that in this case, it does in fact make more financial sense to add new developers as maintainers, rather than start them on a new development effort.

So we can therefore assume that the company has done its risk management, weighed the pros and cons of both approaches, and decided on one. As a developer that aspires to craftsmanship principles it raises a few questions relating to optimizing my personal satisfaction with the job and ideally, mitigating some of the company’s risks in using new developers as maintainers.

• As a software craftsman in a software maintainer role, what can I offer the company, that the original developers cannot?
• How can I be just as valuable and productive to the company as the original developers?
• According to one interpretation of the Pareto principle, 20% of employees are responsible for 80% of the value. How can I ensure that I am in that 20%?

You see, it is not, and should not, be sufficient for me to be a slightly less competent, less effective and less valuable version of the people who wrote the original software. For me the answer would seem to be, find a way to complement, rather than compete with the original developers. It does not make sense to perform the same actions in the same way as the original developers. We have different strengths and weaknesses. Strengths of the original developers, which can be seen as weaknesses of the new maintainers include:

• Extensive domain understanding.
• Extensive understanding of the code base.
• Moral authority.
• Embedded in company culture.

Strengths of new maintainers, which could be seen as weaknesses of the original developers include:

• More recent exposure to other domains.
• More recent exposure to other coding styles and languages.
• More recent exposure to different cultures and ideas in general.
• Fresh start, no company history categorizing or constraining them.
• Not so solidly embedded in the company culture, already liberated from their “comfort zone”.

I was looking for analogies, from historical craftsmanship, to see what general patterns I could notice, but nothing seemed to fit quite right. Usually the thing being designed or maintained exhibit much less complexity and variation than a million line software product, such as shoes, or fountain pen nibs. Often where the product really is complex, such as a car, we start to move away from craftsmanship, and we start looking at software mechanics. Even in this case, a car is still less varied than software, and the mechanic doesn’t have to play around with it for days to discover how it works, he just reads the manual, and looks up tables of symptoms and applies the solution suggested by the manufacturer. That is an idea that could be explored further in the context of software, but I think its usefulness is limited. I could definitely say that a little more documentation would make maintenance easier, but in this case it would cost more that it would bring to produce such documentation, and would suffer the classic disadvantage of always being more out of date than the software itself. It does lend itself to underscore the importance of design techniques such as Domain Driven Design, particularly the use of Ubiquitous Language, to enable source code to be a tool that aids in domain understanding, rather than requiring huge efforts to be understood for its own sake.

One useful analogy I did think of was the boiling frog analogy. The original developers are like frogs placed in cool water that is slowly brought to boil, and the new maintainers are like frogs placed directly in boiling water. The former experiences change slowly over time, and finds it difficult to notice such change occurring, and accepts the current situation as normal, because it is what they are accustomed to. The latter frog notices things straight away, anything that seems not quite right, is immediately apparent.

So I am lead to the following ideas about how and where new developers might get their chance to shine on a maintenance team, alongside the original developers:

• Customer facing tasks and analysis. “I am still new to the travel industry, but in the automotive and health care industries we solve such problems by … have you ever thought of doing it that way?”
• Change and risk management.
• Retrospectives.
• Architecture and software design. “Of course that is going to be difficult to solve in that iterative loop. In my last job we used a lot of LISP and recursion so I believe that if we used recursion here…”
• Things independent of the domain, such as Quality Assurance (not QC/Testing but real QA), Technical Debt management, code reviews, optimization, porting, integration, deployment.
• Representing the company at conferences, and in writing on blogs and in magazines and journals etc, on technical rather than domain related issues.

However, this is not a list that I would hand to the employer and demand that changes are made to suit me. If the employer felt that changes were necessary, he would make them himself. A software craftsman’s career is his own responsibility. He has to carve out his own role, and define the way he will deliver value, within the constraints he his given.

In my case, working within an agile/iterative development process, where we get stories, divide them up into tasks, and tackle them one after the other, it should prove to be somewhat challenging to carve out a role that allows me to answer my three questions above, but a rewarding challenge, one that I, as part of my recent acceptance of my role as maintainer, am quite excited and willing to take on, not so much as an obstacle, but as an opportunity to embrace change on a personal level, and most importantly, to increase the value that I can deliver for my employer.

I recently finished reading Tom Gilb’s ‘Principles of Software Engineering Management’, a book, published in 1988, that I started reading a few months ago for a uni assignment. At the time I was not too impressed. A lot of what is written in the book seemed to take software development in opposite directions to what I have come to appreciate as almost being best practices. Development teams today tend to be reducing the distance to the customer and being formed by tearing down the walls between analysts, designers, developers and testers, and working together to deliver functionality that satisfies customer requirements with minimal waste. The book however defines the following roles: Infotect, Softect, Software engineer, Specialist software engineer, Softcrafter, Moderator and Inspector. According to the book, I, as a “Softcrafter” presumably, would be “a software craftsperson who constructs software according to the design of others.”

But hold on, isn’t that what the compiler does? I am however pleasantly surprised to see the term “craftsperson” used. I suppose I could qualify as a Software Engineer who: “translates required software system attributes into appropriate designs capable of meeting those requirements”. I assume those designs are documentation rather than source code, or what would the softcrafter do?

Much of the book introduces Gilb’s Evolutionary Delivery Method, or Evo, which develops software increments or iterations just like todays agile and lean teams. In this respect Gilb was one of the first to present such an idea. (More information on the history of Iterative and Incremental Development can be read about in this excellent PDF.)

Quality attribute specifications are another worthy aspect of the book. Most of the time requirements (regardless of methodology) are either/or type things. A feature is either present or not. Gilb advocates an additional set of requirement specifications that are expressed in quantifiable measures. They express how well a solution satisfies a requirement rather than simply whether it does or not, and can be tracked over time to see how future changes impact on previously completed solutions. I find this concept could be a powerful tool for setting constraints on both external and internal quality attributes like technical debt levels, that so often get neglected in the relentless pursuit at delivering customer value today (at the expense of delivering it in six months.) Another good article that explores this concept further is Ryan Shriver’s Measureable Value with Agile

Inspection is another interesting topic. The book borrows mostly from the work of Michael Fagan at IBM. I do find it to be rather heavily focused on inspecting for errors in documentation though; errors in the software itself are briefly mentioned as an afterthought. I think in the meantime we have come to realize that documentation is always going to be a vaguer, less correct, lagging version of the software itself, and by realizing this we can minimize waste by focusing on software as design, treating documentation as a temporary guide rather than spending a lot of effort keeping it definitive and up to date when it often never gets read, and by reducing the distance between the customer and the developers, and therefore the need for some much documentation in the first place.

I am however inspired by the use of inspection as a tool for identifying and quantifying internal software quality attributes that can’t be measured easily (yet) with an automated tool such as antipatterns, code smells and technical debt, as well as a learning tool, to help get new maintainers up to speed on large code bases.

Throughout the book, and in the entire last section, one can find all sorts of case studies, tips, and references to other literature.

Overall, I find it worth reading. However don’t forget it was published in 1988, so parts of it may seem old fashioned, or contrary to what people now understand works well in software development, and have been further, and arguably better explored by others in the meantime, particularly in the agile and lean communities. However, theres a lot here, such as quality attribute specifications, that are still today under-discussed, and in those areas, Gilb’s book remains not only relevant but authoritative.

I have started reading another of Gilb’s books, “Competitive Engineering: A Handbook For Systems Engineering, Requirements Engineering, and Software Engineering Using Planguage”. It describes itself more as a handbook, so I am still unsure if I will continue reading it cover to cover, or wait until I my employer implements planguage and I can use it as a reference.

I recently uploaded a couple of old, small, unimpressive open source projects to GitHub.

The first one, PolyDriver, I hacked together in 2003. It is a user space linux driver for PolyTrax powerline modems. I was pretty lucky to be able to convince the boss that Open Source was the way to go for the linux driver. It was a lot more straight-forward than the Windows 2000/XP one too. I don’t have a PolyModem lieing around to really test it, but maybe someone does, and the driver can be useful for them.

The other one, Kurts NetSend, (really needs a better name) is a replacement for the “net send” command that was removed in Vista. I wrote it because my employer at the time used that command to inform everyone of when morning tea was ready, and all the vista users were missing out. It was, and still is, also hosted on codeplex, and was a featured codeplex project on the MSDN Vista page for a while. I think it should work on Windows 7, but have not been able to test it out yet.

They both contain pretty ugly code, the NetSend one could probably be improved a bit, I would start by separating the gui and the backend, and perhaps doing a new WPF front end or something. Otherwise they are both pretty small and could be ideal starting points for someone who wants some practice with refactoring to clean code or something.

A lot of people on the Software Craftsmanship mailing list consider Practice to be not only an important part of Software Craftsmanship, but the main part. I have personally always had a bit of a love/hate relationship with practicing. When I was younger, that was my main way of learning how to program. I would sit there and go through tutorials, and dream up useless little example apps that I could use to explore a particular technology further. As time went on though, and I went through uni, and started full time work, all those technologies that I practiced became obsolete, but more importantly the practice didn’t really seem to teach me the fundamentals of what is really important in developing software: delivering value by understanding the business and communicating and collaborating with people. Sitting at home typing in source code with no value, and outside the constraints that guide the development of real software was fun, but only very superficially satisfying. A crude simile came to mind: it was like trying to learn about romance by masturbating. Sure the raw mechanics seem roughly similar, the technology is partially useful, but you are only really covering a small part of the issue. The entire human aspect is missing, the constraints are not there, all one really does is practices the easy bit: the typing in of source code. In fact, just as masturbation may in fact harm the ability to form real relationships, practicing may harm the ability to create real software (for example one may become so used to the freedoms of practicing alone on his own project, that he cannot deal with the real world constraints of business needs and working with others).

But I was just doing it wrong. Practicing is valuable, provided it emulates as close as possible the real world situation. You need to incorporate the planning aspect, the collaboration aspect, the delivering of value and business realities. Without those practice could do more harm than good.

One step I have taken to simulate planning for example, is the implementation at home, of a Kanban System to help balance the time I spend on practising with that on other household matters.

Each of us, my wife and I have a backburner as well as the Backlog that only shows the 4 most important items. Whoever has the yellow magnet in their backlog takes the next turn to Select an item from their backlog and move the corresponding post-it to the Selected column, provided there is a free slot. (The numbers in each section are the maximum number of items allowed in each section). At that moment the time is written on the post-it. When someone wants something to do, they choose an item from the Selected column, and move it to the In Progress column, writing once again the time on it.  When the item is complete, it will be moved to To Verify, if the item was originally posted by the other person, or directly to Done if you just finished one of your own tasks. Once an item is verified and done, the Lead Time (time from Selected to Done) and the Cycle Time (time from In Progress to Done) can be calculated and can be averaged and used later in optimization and statistics.

It seems to be working ok, although we have only just started using it.

As far as the collaboration aspects go, I have already completed one small task that was inspired by a mailing list message. Someone was looking for a specific example, of a simple calculator application, that could be used to help practice and demonstrate Clean Code concepts. The specific example was unfortunately proprietary, so I took the initiative of starting a GitHub project to develop a calculator collaboratively, ideally helping myself and others to practice using Clean Code concepts.

I originally intended it to be a bit more collaborative, but ended up working on it alone. Still I think it will form a fairly good starting point for future endevours. Such as when learning new tools and libraries, I have at least a fairly simple project that I can play with.

It still doesn’t really deliver that much value though, so I have also joined another online project that may promise to be a fairly good simulation of a real world collaborative, agile project, but it is a bit slow getting underway (which is great really, that’s exactly the kind of issues that one doesn’t get to experience when working alone). Hopefully it will kick off soon and I can write something about it.

Apart from that, there has been some talk in the local use groups about starting weekend long coding dojo type things where we all meet up somewhere and just develop something together. So if that gets going I will be hopefully involved in that.

My current job mostly involves the maintenance of legacy C++ code, so I am really feeling the need for practice to develop the skills I get little chance to develop in my job. So with this new understanding of, and appreciation for practice I am pretty excited about the things I might be learning in the future.

I had a chance on the weekend to catch up on some software development related videos, and thought I would share here what I watched and what I thought of what I watched.

Language Oriented Programming in F# – Roger Castillo

Up until now I had watched and enjoyed several F# related videos and screen casts, and played around with a few tutorials, and I checked this one out for some more deeper insight. It was interesting, but I think I would have gotten more out of it if I was actually working properly with F# rather than just playing around with it. I also don’t think I am such a fan of screen-casts unless I am really trying to learn something specific, and can go through it on my own machine while watching it.

On Being A Journeyman Software Craftsman: Road Thoughts – Visible Metrics

Next on my list was this Corey Haines video, unfortunately I haven’t been able to watch it yet, as I was only available as a streaming flash video. If I try and play any flash video in full screen, the quality is terrible, the CPU maxes out, overheats, and eventually crashes. I have a Firefox extension that can download and convert flash video, but it didn’t seem to work on this site. I do want to check it out, so I will leave it on my list, and see if I can work out some other way of downloading it.

WPF Videos

I still need to learn WPF properly, so I thought watching a few screen-casts might help. I watched one of the basic ones, but like I said above, I really need to go through and do it on my own machine at the same time to really learn anything from it. Also I think when it comes to specific technologies, I would prefer to learn things “just-in-time”, when I have a context in which to think about what I am learning. I think I am more interested in videos when it comes to learning more of the technology-independent fundamentals.

Virtual Alt.NET Videos

I try to attend most of the European VANs, but the classic VAN also has a lot of good guests that I mean to catch up on later. Looking at the Viddler page was quite surprising. Once you include the Aussie VANs and the Brown Bag meetings, there are a lot of videos there! Most of the ones that had guest names in the title I had seen, so I just picked a couple of random ones to skim through. There are definitely some good nuggets of information there, but it is not easy to find.

F# Webcast Series

I had seen part one of this series a while back, and made a note to watch the rest of it. I started to watch part two, but once again, for these technical how-to screen cast type things I think it really requires that we go through and do the exercise along with the screen cast to really learn anything.

But there are exceptions. Over a year ago, I remember watching the Hibernating Rhinos and Summer of NHibernate screen-cast series, and they were so good that I was able to learn a lot just by watching. I think the difference was that I was at the time actually working on a DDD .NET project at work, and was able to directly relate what I was seeing to how I could use it in that project.

I am sure if I was actually using F# or WPF at work I would be able to enjoy watching screen-casts on those topics without going through and doing it locally too.

Anyway, I have made a note to go back and look at these F# screen casts again while actually trying it out myself.

Project Natal

Not sure how this ended up on my list. I am not a gamer, and can’t imagine getting an xbox any time soon. Still its interesting seeing what new technologies are coming out. I remember hearing about it first on the Totally Rad Show, and thinking it was pretty much a gimmick.

The Myth of the Genius Programmer

Now this was great. This is the type of video I can actually sit there and enjoy. I think all developers should continually watch videos like these. It covered topics like working in a team, fear, dealing with mistakes etc. Definitely check this one out.

Artisanal Retro-Futurism crossed with Team-Scale Anarcho-Syndicalism

This video from Brian Marick was probably more entertaining than useful, but I enjoyed it immensely. On the surface it appears to be a plan for restoring the values in the agile world by using some crazy ideas from the past. Also worth checking out.

Videos from the Norwegian Developer’s Conference

I remember hearing a lot about the NDC, and wishing I was there. Now the videos are available. I checked out the The HaaHa Show, but ended up skimming through it as asp.net is not really my thing. I did watch Robert C. Martin – Clean Practice: Agility and Craftsmanship and enjoyed it a lot. He is really both an informative and entertaining speaker. I think I enjoyed his RailsConf “What Killed Smalltalk Could Kill Ruby, Too” talk a bit more. This NDC one seemed a bit rushed and tried to cover a little bit of too many things.

I intend to go back and check out more of the NDC videos.

Haskell and Erlang: growing up together

This keynote from Simon Peyton-Jones was great. Unfortunately the quality is not that good, you can’t really see what is on the screen at all. Simon Peyton-Jones is both very clever and also a pretty funny guy. This video gives a bit of background to how both Haskell and Erlang came to be, a quick overview of how their type systems differ, and the bulk of the talk is about how concurrency works in each language.

I think Erlang might be my language to learn in 2010.

Oh and while we are on the topic of Microsoft Research staff and Haskell, check out some Brian Beckman videos. His video Don’t fear the Monads really helped clear up some of the confusion I had (still have) related to monads.

Software development with Real Options

I have been hearing a lot about “Real Options” on some of the lean development lists lately, so I was curious to see if it was just another way of talking about the “last responsible moment”. Turns out it is, but Real Options takes it a bit further, and helps bring in some ideas from handling risk in finance, and adapt them for software development. This video is a good overview of how Real Options applies to software development.

Windows 7 videos

Somehow these advertisements for windows 7 ended up on my list. I watched one, thought it was stupid. I think it should be sufficient to just read a list of what’s new in 7 and try them out once I guess.

Bindings, Platforms, and Innovation

I found this video to be disappointing. It was an incoherent ramble. Something about stories, and how we all don’t understand something because stories are dangerous, and how stupid we all are for not knowing the difference between the web and arpanet or something. I think he was going for a spontaneous, stream-of-conscious style, but it just came across as unplanned. He would start a sentence, and then restart it several times before actually finishing a sentence, and still not manage to actually say anything. I looked up Bob Frankston afterwards, and he sounded like someone I should be impressed by, but this video did not achieve that.

DDD in a distributed world video

This one from Gojko Adzic was great. I cannot get enough information about DDD. This video really emphasized the importance of aggregates as a business concept, and provided some good technical hints related to handling aggregates in a distributed system.

Project Tuva

This is not actually anything to do with software development at all. It is a collection of Richard Feynman videos about physics. I watched the first one, and it was enjoyable to watch because of Feynman’s personality and the way he puts things across. A lot of people seem to concentrate more an the video player aspect than the content itself, calling it some profound new way to learn or something. Well, the video player does have a lot of extra interactivity features, which is cool, but hardly a new paradigm shift. For me it is more about the content than the player. If I am ever in the right mood I shall go back and watch some more of the videos.

And that is basically all I had time for last Sunday. I did not get through my entire list. I still intend to watch:

• Whatever is new and interesting on Software Development Videos
• Videos from the Miami 2009 Lean & Kanban conference
• Whatever is new and interesting on Dimecasts
• Some more NDC videos
• Gary Short discusses technical debt

Maybe next Sunday…

A lot of the time when reading about software development, I used to end up pretty confused. It just seemed as if the world of software development that I was reading about was a totally different world to what I was experiencing day-to-day at work as a software developer, and when trying to use ideas that I read in my work, I was observing different results to what I should expect. Over time, I have come to understand why this is. It is because the jobs I end up in tend to differ in various ways to what gets typically written about.

In particular I have identified four “axes” along which my experience differs from the typical situation.

• Projects vs Products

A lot of what I learned at uni, the books I read, people I talk to and websites etc, assume that software is developed mostly as projects. That is, they have their own organisational structure (project teams existing alongside functional departments), their own managers and budgets, clear notions of what counts as success and failure, and most importantly a semi-fixed timespan or some date in the future at which the project finishes, or at least enters some phase (often the expensive maintenance phase) at which software development stops.

Apart from a couple of exceptions, almost all of my software development jobs have involved ongoing products rather than projects, no separate teams apart from the software development department, no separate budget, no clear notions of success or failure and no end date. The software will usually be continually updated, and expected to be so permanently. In project terms, such work resembles a permanent maintenance phase.

• In House or Outsourced vs Mass Market

In my course of study, we often compare two types of software development depending on customer involvement. The first is the classic in-house IT development, where the customer is someone, usually another department, within the same organisation, and the second is where a specialist software development or consultancy firm develops software for an external customer. I was never able to make much sense of this, until I realised that except for a couple of exceptions, the software I am typically involved in represents a third main type. Software that is sold on the open market, to potentially many customers rather than a single customer.

When discussing issues involving customer involvement like business and requirements analysis, market oriented software is typically very different to what gets discussed in most books on software development. One major difference is that instead of the customers telling the developers what they want, marketing attempts to convince customers what they need. Often rather than developing software to improve an existing business process, market oriented software attempts to provide whole new processes that often need to be sold as part of the package alongside the software itself.

• Domain Complexity vs Technical Complexity

For a long time I was annoyed at myself for not “getting” domain driven design, or at least not appreciating the power of the ubiquitous language. Until I realised that the DDD books are coming from the classical enterprise IT angle. In such environments it really is possible to talk to the customer in their own language about all levels of the software, not just the problem but the solution as well, from the database to the gui, and to a high level of detail within the business logic itself. This has however not been possible with the types of software I have worked on, where the most we can do with the customer language, or even the customer knowledge, is form a thin wrapper over the true technical complexity.

For example if the customer, say the owner of a security company, wants a software program to match faces on a security camera video feed with a database of stored photographs, then there is not much point getting into too much detail with the customer on exactly how we can solve his problem with a neural network, we should rather talk to other developers who may not know the domain language required to specify the problem, but understand the technical complexity required to implement the solution.

• Cowboy Coding vs Traditionally Managed Development

I would like to think that now-days lightweight and agile methods are making inroads into both “Ad Hoc” (a polite term for unmanaged, or at least non-professionally-managed) and traditionally managed (e.g. waterfall) development efforts, so this distinction is becoming less relevant, but over the last few years as I have tried to improve the way I (and in some cases others) develop software, I have found the advice available less realistic or relevant to the cowboy teams I tend to work in.

Especially earlier on in my career, by bosses were typically not trained to manage software development. They simply knew what they wanted, and requested it to be implemented. Any decision making beyond that, such as whether something should be done or not, how it should be done, or how to improve quality, was simply driven by a mixture of blind hope, guesswork and wishful thinking. Real seat of the pants stuff, I guess it would have been exciting had it not been so frustrating and wasteful.

The difference is most apparent when talking about migrating to agile methods for managing software development. Pretty much everything written about migrating to agile, is talking about migrating from “waterfall” to agile. Trying to use advice that works for waterfall teams, while helping to introduce agile to teams that are not used to a disciplined approach to managing their software development efforts is very likely to fail. It should be obvious really. A team migrating from a traditional to an agile approach needs to scale a heavy process down to size. A team of cowboys needs to build up a “process” (or culture is probably a more appropriate word) where none existed. Following inappropriate advice is likely to have the opposite effect to that intended!

These four axes are not totally independent. I have noticed that projects, for a single customer, than have solutions that can be fairly deeply specified side by side with the customer and have professional management tend to form one “clump”, and the mass-market, cowboy coded, technically complex products that I have worked on form another. The former are basically your classical enterprise business projects, focused on using data to achieve a fairly specific objective, and the way it moves between the database, processing, and user, while the latter is pretty much everything else, including consumer software, device drivers, software sold as part of a hardware package, or to be used to enable a certain profession to work in new ways and solve a wide variety of objectives.

The latter category seems like it should be a much larger, richer, more varied environment, so I wonder why so much that is written seems to assume the perspective of the classical enterprise IT project? I still don’t really know.

Because so much of what is written comes from the perspective of single-customer-driven, managed, projects, yet so much of my experience comes from significantly different environments, I guess it is important to state where I come from so any differences in perspective can be explained. I also hope that future posts will help fill a void, and provide information for developers who work outside of the classical IT environment.

Here are some quotes that I used to have on my wall in my old job. I don’t necessarily agree with all of them, some of them are interesting because they summarize points of view that were once widely held, but now serve to remind us how things change. However some of the oldest quotes are also the ones most valid today.

“Simplicity is prerequisite for reliability “
- Edsger W. Dijkstra

“The process of preparing programs for a digital computer is especially attractive, not only because it can be economically and scientifically rewarding, but also because it can be an aesthetic experience much like composing poetry or music.”
- Donald Knuth

“The perfect project plan is possible if one first documents a list of all the unknowns.”
- Bill Langley

“Complexity kills. It sucks the life out of developers, it makes products difficult to plan, build and test, it introduces security challenges and it causes end-user and administrator frustration.”
- Ray Ozzie

“Copy and paste is a design error”
- David Parnas

“There are two ways to write error-free programs; only the third works.”
- Alan J. Perlis

“Technology is dominated by two types of people: Those who understand what they do not manage. Those who manage what they do not understand.”
- Putt’s Law

“Ugly programs are like ugly suspension bridges: they’re much more liable to collapse than pretty ones, because the way humans (especially engineer-humans) perceive beauty is intimately related to our ability to process and understand complexity. A language that makes it hard to write elegant code makes it hard to write good code.”
- Eric S. Raymond

“You know you’ve achieved perfection in design, not when you have nothing more to add, but when you have nothing more to take away.”
- Antoine de Saint-Exupery, Wind, Sand and Stars

“Without requirements or design, programming is the art of adding bugs to an empty text file.”
- Louis Srygley

“Computer programming is tremendous fun. Like music, it is a skill that derives from an unknown blend of innate talent and constant practice. Like drawing, it can be shaped to a variety of ends – commercial, artistic, and pure entertainment. Programmers have a well-deserved reputation for working long hours but are rarely credited with being driven by creative fevers. Programmers talk about software development on weekends, vacations, and over meals not because they lack imagination, but because their imagination reveals worlds that others cannot see.”
- Larry O’Brien and Bruce Eckel in Thinking in C#

“It is not necessary to change. Survival is not mandatory.”
- Edwards Deming

“Quality is not a project based variable, but a corporate asset.”
- Ken Schwaber

“Adequacy is sufficient.”
- Adam Osborne

“Brilliant process management is our strategy. We get brilliant results from average people managing brilliant processes. We observe that our competitors often get average (or worse) results from brilliant people managing broken processes.”
- A senior Toyota executive

“He who will not apply new remedies must expect old evils.”
- Francis Bacon

I just set up a new google group/mailing list called agile-quality.

Check it out here: http://groups.google.com/group/agile-quality

My initial welcome email says:

I intend it to be a place to discuss all aspects of software quality,
particularly as relevant to or involving agile principles, values or
practices.
This includes things like:

• What is quality?
• Quality Control
• Quality Assurance
• Relevance of quality to project managers/product owners/scrum masters etc
• Quality Management
• Testing (even though there is already a great agile-testing group)
• ISO 9001 Certification
• ISO 9126 Software quality model
• Other standards and models for software quality
• Technical debt and the management thereof
• Metrics for software quality
• Customer involvement in quality issues
• Discussing other “movements” or groups with a focus on software quality
• and agile
• Craftsmanship as it relates to software quality
• Software design; how it relates to software quality
• The Law as relevant to software quality.
• Discussing books, magazines, websites, blogs, quotes etc about software
• quality
• Which agile methods and practices specifically address quality and how?

I expect it to cover technical aspects of interest mainly to developers,
such as clean code design, as well as more managery type topics, such as
QA, certification and legal aspects, and everything in between. Ideally
this could be one mailing list where the two groups can mix, and form
consensus etc.
About me: I am just a software developer (and project management
student) interested in software quality and agile. I am not an author or
consultant (yet/at the moment). I don’t really have much of an agenda to
push except to discuss and learn more about software quality in agile
environments. I do have opinions and they will pop up in later emails.

Currently the list is unmoderated, and I will only change that if
spammers attack. Personal disagreements, teasing, slightly off topic
messages and banter will probably not be moderated away (at least for
now), I think such things help build community. The moderation policy
will be pretty loose but I reserve the right to make changes if it seems
necessary.
Thanks for checking out the mailing list, and enjoy!

I was just reading the XP mailing list and followed a link to a debate in the Agile blogosphere that seems to be a sort of XP vs Scrum type thing. Basically the popularity of Scrum means that people are adopting Scrum and considering themselves Agile without engaging in the appropriate development practices that XP provides. Some claim the direction of Agile needs to change, to put more emphasis on development practices and less on Scrum, or its the end of Agile.

I don’t intend to join in this debate, as I think both sides are wrong, but I will take the opportunity to write how I see it.

The word “direction” is interesting. As if there is something in the distance called Agile thats pulling us towards it. I guess thats how most business processes work. Someone has an idea, and prescribes it in the form of a concrete model that is supposed to be emulated. In such cases this concrete model looks indeed like an endpoint, towards which one finds the direction to be followed.

Agile is, to me at least, a bit different. Its more of a decidedly non-concrete set of principles and values. They don’t define an end point at all, hence no fixed, single direction. They define a starting point, from which more concrete practices and process are pushed out, or emerged. Once a team or an individual develops those principles and values as part of their culture, they exhibit certain behaviors that are influenced by those principles and values. They may or may not end up looking more or less like XP or Scrum. Different interpretations of those Agile principles and values, as well as principles and values outside of Agile, as well as subjective circumstances will mean that whatever emerges, is likely to emerge in as many different directions as individuals and teams that start from those principles and values. XP and Scrum are just head-starts along pretty good directions. I would suggest that a team that adopts Agile practices and processes only for planning while neglecting the development aspects or vice versa has made the mistake of neglecting the principles and values and seeing (some particular sellable concrete package of) Agile practices as a destination to move towards rather than a set of principles and values to guide the emerging of behavior. Hey at least they are a step above those who think Agile and Scrum are TFS templates though!

So rather than watching Agile circle the drains of concrete Scrum (and XP) practices, let it pour out from the values tap of the manifesto and the principles.

Forget about selling Scrum practices (or XP practices) as “directions” for Agile. Sell the principles and values and let the practices emerge, in all sorts of directions!