A lot of the time when reading about software development, I used to end up pretty confused. It just seemed as if the world of software development that I was reading about was a totally different world to what I was experiencing day-to-day at work as a software developer, and when trying to use ideas that I read in my work, I was observing different results to what I should expect. Over time, I have come to understand why this is. It is because the jobs I end up in tend to differ in various ways to what gets typically written about.
Image courtesy of “e g g” on Flickr
In particular I have identified four “axes” along which my experience differs from the typical situation.
- Projects vs Products
- In House or Outsourced vs Mass Market
- Domain Complexity vs Technical Complexity
- Cowboy Coding vs Traditionally Managed Development
A lot of what I learned at uni, the books I read, people I talk to and websites etc, assume that software is developed mostly as projects. That is, they have their own organisational structure (project teams existing alongside functional departments), their own managers and budgets, clear notions of what counts as success and failure, and most importantly a semi-fixed timespan or some date in the future at which the project finishes, or at least enters some phase (often the expensive maintenance phase) at which software development stops.
Apart from a couple of exceptions, almost all of my software development jobs have involved ongoing products rather than projects, no separate teams apart from the software development department, no separate budget, no clear notions of success or failure and no end date. The software will usually be continually updated, and expected to be so permanently. In project terms, such work resembles a permanent maintenance phase.
In my course of study, we often compare two types of software development depending on customer involvement. The first is the classic in-house IT development, where the customer is someone, usually another department, within the same organisation, and the second is where a specialist software development or consultancy firm develops software for an external customer. I was never able to make much sense of this, until I realised that except for a couple of exceptions, the software I am typically involved in represents a third main type. Software that is sold on the open market, to potentially many customers rather than a single customer.
When discussing issues involving customer involvement like business and requirements analysis, market oriented software is typically very different to what gets discussed in most books on software development. One major difference is that instead of the customers telling the developers what they want, marketing attempts to convince customers what they need. Often rather than developing software to improve an existing business process, market oriented software attempts to provide whole new processes that often need to be sold as part of the package alongside the software itself.
For a long time I was annoyed at myself for not “getting” domain driven design, or at least not appreciating the power of the ubiquitous language. Until I realised that the DDD books are coming from the classical enterprise IT angle. In such environments it really is possible to talk to the customer in their own language about all levels of the software, not just the problem but the solution as well, from the database to the gui, and to a high level of detail within the business logic itself. This has however not been possible with the types of software I have worked on, where the most we can do with the customer language, or even the customer knowledge, is form a thin wrapper over the true technical complexity.
For example if the customer, say the owner of a security company, wants a software program to match faces on a security camera video feed with a database of stored photographs, then there is not much point getting into too much detail with the customer on exactly how we can solve his problem with a neural network, we should rather talk to other developers who may not know the domain language required to specify the problem, but understand the technical complexity required to implement the solution.
I would like to think that now-days lightweight and agile methods are making inroads into both “Ad Hoc” (a polite term for unmanaged, or at least non-professionally-managed) and traditionally managed (e.g. waterfall) development efforts, so this distinction is becoming less relevant, but over the last few years as I have tried to improve the way I (and in some cases others) develop software, I have found the advice available less realistic or relevant to the cowboy teams I tend to work in.
Especially earlier on in my career, by bosses were typically not trained to manage software development. They simply knew what they wanted, and requested it to be implemented. Any decision making beyond that, such as whether something should be done or not, how it should be done, or how to improve quality, was simply driven by a mixture of blind hope, guesswork and wishful thinking. Real seat of the pants stuff, I guess it would have been exciting had it not been so frustrating and wasteful.
The difference is most apparent when talking about migrating to agile methods for managing software development. Pretty much everything written about migrating to agile, is talking about migrating from “waterfall” to agile. Trying to use advice that works for waterfall teams, while helping to introduce agile to teams that are not used to a disciplined approach to managing their software development efforts is very likely to fail. It should be obvious really. A team migrating from a traditional to an agile approach needs to scale a heavy process down to size. A team of cowboys needs to build up a “process” (or culture is probably a more appropriate word) where none existed. Following inappropriate advice is likely to have the opposite effect to that intended!
These four axes are not totally independent. I have noticed that projects, for a single customer, than have solutions that can be fairly deeply specified side by side with the customer and have professional management tend to form one “clump”, and the mass-market, cowboy coded, technically complex products that I have worked on form another. The former are basically your classical enterprise business projects, focused on using data to achieve a fairly specific objective, and the way it moves between the database, processing, and user, while the latter is pretty much everything else, including consumer software, device drivers, software sold as part of a hardware package, or to be used to enable a certain profession to work in new ways and solve a wide variety of objectives.
The latter category seems like it should be a much larger, richer, more varied environment, so I wonder why so much that is written seems to assume the perspective of the classical enterprise IT project? I still don’t really know.
Because so much of what is written comes from the perspective of single-customer-driven, managed, projects, yet so much of my experience comes from significantly different environments, I guess it is important to state where I come from so any differences in perspective can be explained. I also hope that future posts will help fill a void, and provide information for developers who work outside of the classical IT environment.