Kurt's Blog

January 27, 2012

Book Review: Scaling Software Agility: Best Practices for Large Enterprises

Filed under: agile, Book Review — Kurt Häusler @ 7:51 am

I just finished reading this very enjoyable book. For me it is always interesting to see how others view agile, particularly in areas that I haven’t thought of much, as it provides me an opportunity to I guess evolve my ideas a step further. I wrote a lot of notes this time, so there should be plenty of material for an opinionated review. The first thing I should point out is that it was released in 2007, which doesn’t sound that long ago, but a lot has changed since then, so it is important to keep that in mind.

SSA

The book starts off by introducing agile, criticizing waterfall, and going into some more depth on specific named agile approaches. I went through this reasonably quickly as most of it wasn’t new to me. I did not know anything about DSDM though, despite living in Europe for 10 years, so it was nice to read about that.

Things get interesting around chapter 7. The Essence of Agile. I was impressed by the initial table about changing paradigms. It reminds us how fundamentally different the changes that need to be made are, at every level. We clearly see waterfall development on the left, and the idea is to move towards agile development on the right. My attention was immediately drawn to the row headed Management Culture. Under waterfall he writes Command & Control. Under Agile he writes Leadership/Collaborative. I am glad he did. I have been making the connection between waterfall development and command & control management before but wasn’t sure if it was quite right. Despite what many in the Kanban community think, there is a bad old management culture we are trying to move away from, and it is ok to call it command & control, and there is a better alternative that we are moving towards based on leadership and collaboration. And to me that is the fundamental context for any organization adopting agile. Agile might just be about software development, but it belongs to a larger, non-software specific packet that involves a fundamental change to the management culture. If people reject that part of it, or don’t understand it, then success will be prevented. In fact I would have almost done the diagram differently. With column headings “Command & Control” and “Leadership / Collaborative”, and had waterfall and agile as mere cells in the table next to a “software development style” row-heading. To me the management culture just seems bigger and more fundamental than the way software projects are planned and delivered. Particularly in the larger organizations that this book focuses on that probably do a lot more than just develop software.

Another thing I noticed in the table was the idea that in agile, teams bid stories. That is a concept I have not heard of before, or understand, so I intend to look into that further.

One thing he mentioned that I disagreed with was to do with the iron triangle. He wrote that waterfall fixes scope but resources and date are flexible, whereas in agile resources and date are fixed, but scope is flexible. My opinion differs in that waterfall-style (perhaps I don’t mean the theoretical waterfall model, but the approach that command & control organizations use to develop software) projects attempt to fix all three sides, whereas agile insists in at least one side being flexible. I find it perfectly acceptable that a fixed scope project is to be delivered using an agile approach, provided there is some flexibility in the delivery date, and of course costs.

Then he mentions that the heart-beat of agile is the time-boxed iteration. This would have been fine in 2007, but now we have the whole flow based development thing, and especially Kanban showing us that continuous flow can be just as, if not more agile, than iterations. A lot of the rest of the book is based on this assumption that agile implies iterations, which compromises its usefulness these days unfortunately. This is especially apparent in the section on releases. Sure meetings should always be time-boxed, but for me the concepts of cadence and flow have replaced the time-boxed iteration, providing even more flexibility and predictability.

One major premise of the book is that agile is indeed capable of “scaling to the enterprise level”. For me agile, or at least named agile approaches, have always been more appropriate for medium-sized and larger organizations as smaller ones tend not to have the problems that agile methods solve. If you are a small startup, perhaps a group of hackers working on a web based app, communicating very well informally, spending less than 10% of your time in meetings, and delivering value then you really don’t need to go throw Scrum in the mix. You already have what larger companies are looking for when they try agile. Now I wasn’t there in the beginning, but it always seemed to me that agile would be something interesting to try if I were ever in a large enough company to justify it, and from what I read a lot of the proto-agile projects where agile methods were developed were in fact in larger companies. As well as that, whenever I heard the phrase “enterprise agile” I thought of scaling complexity and focus down from the enterprise level and projects down to the story and team level. The idea of “scaling agile up” seemed contrary to the whole point, and almost an oxymoron. The misconception I had when hearing about “scaling agile up” was it is all about compromise. How can organizations get some of the benefits of true agility when they aren’t able to, or prepared to make the necessary changes. This is mostly a philosophical point, but it might be useful to explain how I was thinking as I begun reading the book.

That impression was mostly reinforced by chapter 8, and I started to worry. Not only were we talking about larger organizations (where I assumed at least the teams themselves could be small and autonomous), but teams with several hundred people. To me that is not even a team.

The next section restored my faith with the phrase “Agile is not a trivial fix, or tune-up, for what we have been doing. It is a radical change in practice and culture”. I love that for two reasons. The Kanban community, of which I consider myself a part, likes to downplay how radical it is, claiming it can in fact be done like a series of small tune ups. This might be be true for some organizations that already have a few key cultural aspects in place, but in general it is as radical a change as any other agile method. The second reason is that it is refreshing to see a word like radical in a book aimed at big serious business. I think a lot of coaches etc. try and downplay the radical nature of the changes that are about to take place, and using words like radical and observing the reaction is a good approach for seeing if key stakeholders are even ready for it.

Then he defines the component team. He actually mentions that the name “component” isn’t that important, and the teams could be focused on different things. When I think of a component I think of something that isn’t that huge, and is developed once, and from then on the team is stuck maintaining that component. I much prefer the term “cross-functional team”. And I prefer to see their focus or specialization emerge, as work gets allocated to them (or selected by them) on a variety of criteria, such as capacity, speed, cost, special skills, level of competence required, perhaps a team wants to build up experience in a certain area, or avoid a knowledge silo etc. A good manager will balance the need to expose the team to new stuff that they have to learn, with the need to get things delivered by experienced teams quickly. Over time it may appear as if the team always does work from a certain set of customers, or indeed always on a group of related components, or always does the really urgent stuff, or the critical stuff or whatever. But there are a variety of approaches that could be used. It probably depends on the type of software being developed and where the requirements come from etc.

One issue I have with teams allocated to components is that I could imagine a story, or many stories, requiring changes to several components. This would require too much communication and coordination overhead between teams. Any team should be able to do most of the work itself on any story they might end up with. You don’t want to force a constraint that all stories have to be isolated to a single component, as it requires too much technical knowledge of the architecture during story splitting. In fact during story splitting it might not even be known which components are involved. The story is really just the problem from the users point of view and shouldn’t require any technical knowledge at all to estimate, prioritize, split or write acceptance criteria for.

As far as teams go, they should be cross-functional, and they should be permanent. Projects should be allocated to existing, already gelled teams, rather than forming a team around a project. (This belief was strongly reinforced by reading The Clean Coder, which I intend to review sometime.)

And yes, if such teams don’t exist, perhaps because everyone is in functional teams within functional departments, then these inter-departmental, cross functional teams, or component teams as the book calls them, would have to be created. (The book also uses the term define/build/test team, which is better than component, but kind of over specifies the degree of cross-functionalism required. In my opinion a cross functional team should cover the whole value stream. Whether that is a subset or superset of define/build/test is context dependent).

I like that mentioned that the PO should not just prioritize customer-value-stories, but also refactoring and was it infrastructure or architecture stories. However with 2012 understanding, we would know that refactoring, infrastructure, and architecture happens, and should normally only happen as part of customer value stories. I would reword it to say that the PO should prioritize the payback of technical debt “stories” alongside customer value stories. Which is the same thing basically. A pretty minor point. I almost feel pedantic bringing it up.

I thought maybe the term technical debt was not widely used back in 2007, and that a similar book today should probably have a whole chapter on it. He does actually use the term once, later in the book I believe.

Now we come on to the topic of release planning. I actually found a lot to disagree with here, but later on in the book he actually elaborates and ends up moving closer to how I view the topic of releases etc. I am not sure if teams should be involved in release planning the way it is described in the book. I think as soon as everyone gets together and allocates specific features to specific release dates in the future there is too much pressure to stick with it. Stakeholders with a poorer understanding of agile might interpret as a promise, or might judge the team unfairly every time the plan is updated. Teams may feel under pressure to stick to the release dates no matter what, and get everything in, burning out and compromising on quality etc. I actually go so far as to not specify release dates at all. Just have a product backlog, and teams should be focused on the current iteration, and regular backlog grooming where stories for maybe the next 1 or 2 iterations can be split, estimated, and have acceptance criteria written for them.

Release planning, at least in Scrum, should be something the PO does almost in secret, by looking at the backlog, and the very roughly estimated, and not yet split stories, and combining that with the velocity to get an extremely vague idea about which stories may be finished in which iteration. If this is communicated to stakeholders, that vagueness has to be emphasized! I don’t think this is a weakness of Scrum, but for those looking for more predictability, then I recommend using Kanban, where we can use actual measurements rather than estimates. And no way should the team be expected to delivering certain features by certain dates, especially at the release level. It is hard enough to commit to an iteration, which is why the Scrum Guide has actually removed the concept of commitment and I believe replaced it with the concept of forecast.

I was always the black sheep that refused to commit to delivering the selected stories when we used Scrum at my previous employer. We didn’t use story points or velocity, work was allocated by available man-hours, and we never completed all selected SBIs. In fact “done” required testing by people outside the team, and usually occurred 2 sprints later! I claimed it would have been unprofessional for me to commit to anything given these factors, but everyone else was like “don’t be a dick, commit just means commit to give it a go and do your best, it is not a promise or anything”. I refused to use a word as strong as commitment for something so wishy-washy as “giving it a good go”.

I was very glad when the commitment was removed from the Scrum Guide. Bob Martin actually talks about the difference between an estimate and a commitment, and I agree with it 100%. A commitment is a promise. It is something that will happen. The most I think a professional developer can commit to is beginning, or working on a task. Not completing it, unless it is a small well understood task. Everything else is an estimate or a forecast. Throwing around commitment like a used tissue is just unprofessional, and reduces trust.

Needless to say. I do not support teams “committing” to delivering certain features in certain releases. We shouldn’t forget it is a PO or management responsibility to ensure that the best possible product is delivered by a certain date, and this happens automatically when the backlog is well groomed and prioritized.

I can’t remember who mentioned this, but I heard once that releases are not technical events but marketing ones. The development teams shouldn’t care less when marketing decides to take a snapshot of the currently releasable software and label it a release, perhaps in response to some market event or need. They are too busy focused on the current iteration. But the book comes back to this view later on, which is cool.

Then he started talking about estimating tasks, and assigning them to people, and I realized he was using a fairly old, and inferior, version of Scrum, where things happen in a different order. Some people, like Ron Jeffries don’t even break stories into tasks, in fact, the Scrum Guide has removed the concept of Sprint Backlog Items! It is actually the PBIs that get estimated, during backlog grooming. In the first part of sprint planning the highest priority ones are selected for the sprint according to velocity, and then the team begins to think about the solution in the second part of the planning meeting. Please use the better version of Scrum as defined in the latest Scrum Guide, it is much better than the old version, which has some problems, as described in the book.

The book doesn’t mention backlog grooming, which I consider an omission. In fact backlog grooming should have more time spent on it than any other Scrum meeting. I would say for a larger enterprise, even more time should be spent on backlog grooming, as the initial stories are likely to be larger, and require more splitting etc. The Scrum guide says no more than 10% of a teams time should be spent on backlog grooming. I would use that 10% as a good guideline for an enterprise Scrum team. That is a half-day per week, or a whole day for a 2 week sprint.

One interesting part compared a waterfall scenario with an agile one. In the waterfall scenario everything is planned years in advance, so a salesman is forced to say “no” to an important customer who needed a quick turnaround time. This was contrasted with the agile team who were able to shuffle things around and deliver the important parts on time. What I thought funny was that the salesman was actually professional enough to say “no” to an important customer. In my experience, the salesman always says “yes”, pockets the commission, and uses pressure to force the team to deliver the software in full, on time, without affecting the existing plans.

I would almost be happy to work for a waterfall organization where the salesmen knew enough about the capacity of the team, and were professional enough to say “no”.

We really don’t say “no” often enough in software development.

He mentions that agile requires forced prioritization. Well I think that has changed too. Scrum now only requires ordering, and with Kanban we don’t even order. We simply select the most important thing to do next during queue replenishment.

Another thing I disagreed with was his suggested that for new teams, “individuals work out their expected velocities” and then combine them to get an initial estimated team velocity. That is total nonsense. What is the velocity of a BA? Of a tester? How much working tested software can they deliver as individuals? It is a team measure. It can only be measured, and not estimated. If you want to do some release planning that requires velocity, you have to do one iteration first, measure velocity, then you can do an initial release planning.

A minor quibble I had was his combining Agile/Scrum Master and Team Leader in a single role. I think these are totally different roles. A team leader, assuming you need one, which is a big assumption, might allocate a task to a junior member, a Scrum Master wouldn’t. A team leader might have some authority that the SM doesn’t have. A SM is busy removing impediments to the team, and spreading agility throughout the organization. The team leader is busy developing software.

He has a “requirements pyramid” where he divides the problem domain and solution domain. For him Needs, are in the problem domain, whereas Features and Requirements are in the solution domain. I disagree with this, although it might be a minor philosophical point without consequence. For me all three are in the problem domain. For me we only start to cross into the solution domain when the team begins to discuss the solution in the planning meeting. We are only fully in the solution domain when we have actual written code.

Actually it is more than philosophical. If you think about software development, it is the process of solving problems with software solutions. His pyramid suggests that the problems are actually solved long before they get to the development team. That whoever breaks down a need into a list of features is solving the problem, and that solution is merely being thrown down the chain for the coders to type in. It is a demotivating model. The problem is a problem, right up until the team solves it. They might split the problem into smaller chunks, but they are just smaller problems not solutions.

He also has a solution continuum, where he presents a choice between a minimum solution that “provides minimum benefit but is easiest to implement” and “fully featured solution that delivers additional value”. This doesn’t make sense to me either, and indicates a poorly groomed story. The developer should always implement the simplest possible solution that satisfies the acceptance criteria, and no more. Of course the team may discuss which of a selection of implementation choices is in fact the simplest, but the amount of value that can delivered should be fixed.

There is a great section on decoupling releases from development, which I think could replace the earlier section on releases and release planning.

The “Changing the Organization” section is probably the best and most important. I had not heard of the Zen concept of “Ba” before. I think learning more about it, and discussing it with senior management and observing their reaction is a great idea to predict the level of success with agile to expect. “Creative Chaos” is another such term mentioned in this section. Any manager than smiles and nods at such terms rather than recoiling away has potential to make the most of what agile can provide.

I like the idea of management being organized into teams as well, with their own impediment backlogs. The idea of giving an aptitude test at the beginning to discover if the right attitudes to change are in place also makes sense. The book suggests more training too, in areas like product development and engineering. I would add general modern management training to the list.

Although often hiring or promoting already trained people is a good option. After all one principle I believe in, is you have to trust people to work in the way they see fit. If you hire a traditionalist manager, you have to allow him to work in a traditionalist way. Training him up in agile and expecting him to work in a way he doesn’t believe in is demotivating and disrespectful. It would be as bad as shipping me off to MS-Project training and expecting me to do waterfall planning. Most organizations have people hidden away in teams that already have the right attitude and skills, and are eager to get a chance to put them into practice. Many developers have learned a lot about agile management as a side effect of their interest in agile technical practices. I would rather promote them to a management role than try and train a traditionalist manager to work in a way he finds uncomfortable.

The section on measurement wasn’t as bad as I thought it might be. A lot of useful suggestions, and nothing too harmful.

So to conclude we have a great recipe book. I would hope people don’t apply all the recipes in advance. It is mostly fodder for chefs to consider while developing their own recipes custom made for their organization. I actually think most standard agile approaches like Scrum and Kanban are perfectly fine to start with and allow an organization of any size to discover what particular changes should be made to handles issues of scale. This book, or an updated version of it, could be a useful companion in that process. I certainly wouldn’t go picking and choosing and implementing wholesale the actual methods inside, in advance, because a big enterprise thinks they will need it. The book actually mentions as much towards the end which was good. It could do with an update though, addressing all the points I mentioned above.

Advertisements

3 Comments »

  1. Wow, that’s a big review! A lot of your opinions resonate well with me, Kurt. Gave me some ideas. Thanks!

    Comment by T. Ferris Nicolaisen (@tfnico) — January 30, 2012 @ 3:42 pm

  2. […] Book Review: Scaling Software Agility: Best Practices for Large Enterprises (kurthaeusler.wordpress.com) Share this:TwitterDiggEmailPrintLike this:LikeBe the first to like this post. This entry was posted in Ukategorisert and tagged Agile management, agiledevelopment, Iterative and incremental development, Scrum, Software development, Software development methodology, Software development process, Waterfall model. Bookmark the permalink. ← Agile methods do not fit every software development project […]

    Pingback by The success and failure of agile methods in software development | Svein Minde Blog — February 24, 2012 @ 12:48 pm

  3. […] Book Review: Scaling Software Agility: Best Practices for Large Enterprises (kurthaeusler.wordpress.com) […]

    Pingback by Agile methods do not fit every software development project | Svein Minde Blog — March 29, 2012 @ 11:55 am


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: