AvailAgility
Karl Scotland – Using Agile to Deliver Value
Lean
Systems Thinking, The Vanguard Method and Software Development
Feb 25th
I’ve recently read John Seddon’s “Freedom from Command and Control“, which introduces his approach to Systems Thinking – the Vanguard Method. A few key points really struck home with me and helped clarify my thoughts on some of the challenges I’ve come across recently.
First is the following simple diagram, which shows that Management is responsible for defining the System, which is ultimately what defines Performance. Management’s role should be to analyse Performance, and change the System to improve it.
Agile initiatives are usually begun in order to improve performance. Seddon says that in order to analyse Performance, we first need to understand the Purpose of the System. Then we should create Measures to provide knowledge of how well we are meeting that Purpose, before finally applying a Method which meets that Purpose, using the Measures to help refine the Method.
Finally, Seddon says that understanding Purpose involves understanding Demand, and in particular, Seddon introduces the concepts of Value Demand and Failure Demand. Value Demand is what do our customers ask us to do because it add value, and Failure Demand is what our customers ask us to do because we failed to something, or do something right in the first place.
I’m increasingly aware of Lean and Agile methods being used for the sake of it. While Scrum, XP and Kanban will generally serve common software delivery Purposes, such as delivering real benefits and in short timescales, using them without recognising the Purpose will often result in no real improvement in Performance. Lean and Agile methods are often used just as alternative delivery approaches within an existing System, rather than as means to change the System itself. These organisations can be thought of as “wall-dwellers”, using a method within existing boundaries, rather than “wall-movers”, moving the boundaries to help create a System which helps meet the Purpose. To quote (or paraphrase) Russell Ackoff, this is “doing the wrong thing righter, rather than doing the right thing”.
Do you know what the Purpose of your software development process is? Do you have Measures about capability against that Purpose? Do you know what your Value and Failure Demand is?
Process Safeguards and Ski Slopes
Jan 8th
One of the joys of working as a coach for EMC Consulting are the regular opportunities to have deep conversations on various topics with my colleagues when we are in the office together. For example, earlier this week myself and Simon Bennett began to discuss way of talking to our clients about process such that we can guide them towards the most appropriate choice. However much we may have our preferred approaches, they may not always be the best solution in any context. In the course of the conversation we compared various styles including waterfall, scrum and kanban, and looked at them on various scales, including risk, reward and control. None of these seemed satisfying as a language to use, until Simon suggested safety.
Different processes have different levels of safeguards built into them. Safeguards are used by processes to compensate for low skill, low maturity or low trust, but they also reduce speed or reward. With a waterfall process, the heavy up-front documentation is a safeguard to compensate for low trust (and often low skill and maturity). When a customer does not want to collaborate, then waterfall provides safeguards aimed at avoiding them rejecting the final delivery (not that those safeguards always work!). At the other end of the spectrum, extremely productive teams often have very few safeguards because of the high trust, collaboration and capability. Those safeguards that are in place (such as automated tests) are very different. Not having safeguards doesn’t make the process dangerous. It means that team doesn’t need them. So when we talk to customers about process, we can talk about the appropriate safeguards needed for the amount of trust that exists, the capability of the team, the desired delivery speed etc.
While exploring this idea, I became aware that at various points I was referring to kanban at different places on the safeguard continuum. At the high safeguard end was a kanban based process that looked like waterfall, but was limiting WIP and reducing queue and batch sizes. At the low safeguard end was a kanban process with extremely low WIP, no estimation and pure pull. That’s quite a wide range of processes. Similarly, we had Scrum down as a range on the continuum. Towards the high safeguard end, Scrum-but changes Scrum to add safety. Towards the low safeguard end, a highly productive Scrum team will just be using the raw essence of Scrum. Even at this end, however, the Sprint boundary in Scrum could be seen as a safeguard. It’s the opportunity to pause and reset if things are going awry.
In trying to articulate the different types of kanban system in terms of safety, I gave a nod to XP and labelled low-safety kanban “Extreme Kanban”, and high-safety kanban “Safe Kanban”. Not entirely happy with this we bounced some more ideas around and came to the Ski Slope metaphor. “Extreme Kanban” became “Black Kanban” and “Safe Kanban” became “Blue Kanban”. A further iteration led us to “Off Piste Kanban” and “Nursery Kanban”. The ski slope metaphor works quite well at a number of levels. Its a nice way of looking and risk and reward. As a beginner at skiing I don’t want to take much risk, but can get some reward on the nursery slopes. As my capability improves, and as I trust my ability (and instructor) more, I’m prepared to go onto more difficult runs and take more risk to get more reward. Nursery slope groups are generally more controlled, staying in a single file group. As capabilities improve, individuals are given more freedom to try things out for themselves. If I were ever to become an expert, I’d be happy to go off piste to get a high reward. Additionally, when I get to a level where I go off piste, I probably don’t need an instructor anymore, but I may need a guide.
Using the ski slope metaphor allows us to decide what level of process to start a new team with based on how good they currently are. We may want to stretch them a bit, but we may not want them to go straight onto a black run process. We also want the team to be able to move up to more challenging but rewarding levels as they improve. This is one of the reasons I prefer a kanban based approach. I believe it allows teams to start with an safer process to begin with and move to more rewarding processes later by removing or changing safeguards, as opposed to pushing teams straight into a process with less safety before they are ready.
Kanban Trail Markers
Nov 24th
When talking about Kanban Systems for Software Development, I always try to emphasis that the Kanban System is more than the tool, and is a System that should be owned by the team, rather than being imposed upon it. By owning it, and being part of creating it, a team are more likely to evolve the system as part of continuous improvement. Recently, I tried a new analogy to make this point, and while its not perfect, I think its good enough to blog about for further feedback.
The Primary Practices of Kanban that I describe, are not Boolean practices that teams either do or don’t use. Rather they are practices that teams should be constantly revisiting, and re-implementing, as they improve and their context changes. To recap, the primary practices I talk about are:
- Map the Value Stream
- Visualise the Value Stream
- Limit Work in Progress
- Establish a Cadence
- Reduce the Kanban Tokens
I currently describing these practices as like trail markers on the team’s journey of process improvement. At any point in time, the team’s Value Stream, Visualisation, WIP Limits, and Cadence identify where the team currently is, and when combined with Kanban Reduction, point the way forward. As when on a hike, only the group knows where it is, and only the group can decide where it should be going. The hiking group must work together, helping each other out and making sure that everyone reaches the destination. If you’ve read Eli Goldratt’s “The Goal” you’ll recognise the comparison to the one he uses.
Where the analogy breaks down is that a team’s trail markers are only ever unique to that team. They cannot be followed by other teams. Instead, they will only show the journey the team has been on in their quest for success.
Outcomes and Sync Steps
Nov 5th
I met up with Jean Tabaka last week for a coffee and we chatted over various things, including Lean, Kanban, “The Don”, Tufte, and Systems Thinking. One of the other areas was around the origins and original intents of Scrum. Jean mentioned an early paper(*) by Jeff Sutherland, written before the current terminology became standard, where he described his process in a very simple way
- Decide and agree on an Outcome, in terms of working software
- Use daily Sync Steps to decide how to best achieve the outcome
Since we had that discussion (which also touched on GTD), I have found that this a great a way of describing the essence of Agility. Inevitably it has also triggered thoughts on ways of comparing and contrasting Kanban with more traditional Agile methods such as Scrum.
In a Scrum based approach, the Outcome is defined by the Sprint Goal, with the Sprint Plan being a means of making a commitment to the Outcome. There is a single Outcome, which is constrained by the length of the Sprint. The Sync Steps are provided by the Daily Stand Up Meeting.
In a Kanban based approach, the Outcomes are the limited work items in the system. There can me multiple Outcomes in progress at a time, but the WIP limits constrain the Outcomes. Planning, and as a result any commitment (or SLA) on the Outcomes is done per Outcome, and Just In Time. The Sync Steps are usually provided by the same Daily Stand Up Meeting, but could also be formed by other forms of Cadence.
So both Scrum and Kanban focus on team Outcomes, with regular Sync Steps to achieve the Outcomes. The way in which those Outcomes and Sync Steps are managed are different.
(*) Unfortunately, I don’t have a reference to this paper. If you recognise it, please let me know!
Update: Jeff actually calls them SynchSteps. References can be found in this 2001 paper, and in early emails. He also refers to Outcomes as Mutations.
A New Lean And Agile Picture
Oct 21st
David Harvey posted a brilliant piece on his blog entitled “The Scrum Picture is Wrong”. I highly recommend reading it. His ideas and suggestions for an alternative Scrum picture got me thinking about how to visualise Lean and Agile software development in a process or label agnostic way. David’s picture looked like a figure of eight, and there seemed to be 2 inputs (a vision and reality), and 2 outputs (the product and the team). A quick google found me what I was looking for here – a figure of eight with a bar across the top and bottom. “A time sign that according to Diderot’s Encyclopedia meant hour for the chemists in eighteenth century France”. That seems quite appropriate. An hour would be quite a good interval for a feedback loop 
This is what I came up with. A vision and reality come together to produce a product and a team through process and improvement (and process improvement)
Now I just need to find a good label to stick on it…
Is Kanban A Relabeling of Scrum?
Oct 20th
Firstly, this post is not an attempt to be divisive or competitive. Instead it is meant to be exploratory. What would it mean for the statement in the title to be true? Actually, the full statement was “People have so misunderstood Scrum, that they’ve reinvented it and called it Kanban”. It was made by Jim Coplien at Scan-Agile, after (but not necessarily the result of) a conversation over dinner where myself and a few others were describing how we used Kanban. Each time we described different aspects of our processes, Jim would say something along the lines of “but I do that with Scrum”.
So what would it mean for people to have so misunderstood Scrum that they have reinvented it and called it Kanban?
- It might mean that at their heart, Scrum and Kanban have the same intent. That both are really focussed on helping teams think about their processes in order to help them succeed.
- It might mean that the way that Scrum was originally articulated (and is still articulated according to the latest Scrum Guide) was not as clear as it could have been. That teams might misconstrue the focus on roles, meetings, artefacts and rules.
- It might mean that there are alternative ways of articulating the same intent. That teams might find alternative articulations valuable.
- It might mean that Kanban can be equally misunderstood. That teams might be bewildered by a less prescriptive approach.
- It might mean that we should spend more time on understanding how to help teams solve their problems and less time arguing and fighting over preferred solutions.
These are just some of my thoughts. They do not mean that I think Scrum is bad – just not perfect. They do not mean that I think Kanban is perfect – although it’s currently my first language. The topic drew a good crowd at the Scan-Agile Open Space and we had a good discussion. What else might it mean?
Evolving a Workflow
Oct 9th
Mary Poppendieck gave a talk on Workflow is Orthogonal to Schedule at Agile2009, during which she very neatly transitioned a schedule-focused view of work, into a flow-focussed view. At least I thought it was neat, so I’m going to try and reproduce the basic elements here, using my favourite agile workflow.
4 Week Time-box Schedule
Here we have a schedule showing an average of 4 features being delivered every 4 week time-box. Each time-box is preceded by 2 weeks understanding the next features, and a release is made every 8 weeks. Its not a bad schedule. There’s a regular release every two months which is better than a lot of projects, but it could be better.
2 Week Time-box Schedule
Now we reduced the time-box to 2 weeks, meaning that we do an average of 2 features in each time-box. This means that the preparation now takes only 1 week. Additionally, we are now releasing at the end of every time-box, which is also reduced to 1 week. Much better.
One Piece Flow
If we continue the evolution we end up working on a single feature at a time and releasing it immediately. Each feature only takes a week to build, and preparation and release times are now down to 1/2 a week. At this point, we don’t really have a schedule anymore, but a natural workflow.
Cumulative Flow Diagrams
One of the things that strikes me about the diagrams above, is that each step in the evolution transforms them more into a smooth Cumulative Flow Diagram. In fact, in the same presentation, Mary showed the following picture taken from Building the Empire State. This is the building schedule from from around 1930, and itself looks remarkably like a CFD. Another example of how we are not inventing anything new, but can learn from other industries and successes.
#lkuk09 – A Tweetrospective
Oct 6th
I ended up making notes at the Lean & Kanban UK Conference with good old fashioned pen an paper. Rather than try and write up those notes into something coherent and meaningful, I have decided to write them up in the style of a twitter stream. These are the things I would have tweeted if I’d been on my laptop. The quantity of “tweets” in no way represents the quality of the presentations. I also make no promises that all of the following “tweets” are actually <= 140 chars!
Monday
Mary Poppendieck – The The Tyranny of the Plan
- Plans – what did construction do before computers?
- Eliminate design loops by consulting experts early
- Design for decoupling workflow
- Cash Flow Thinking – Cost of Delay
- Design based on constraints of situation
- Establish a reliable workflow 1st
- Schedule is orthogonal to workflow
- Build schedules based on experience, not wishful thinking
- Variance from plan is a learning opportunity, not a performance failure
- Reliable workflow – output, pathway, connection, method, improvement – Steven Spears, Chasing the Rabbit
- Polaris Success – Quality of Leadership, Focus on deployment, Decentralised Competitive Org, Emphasis on Reliability, Esprit de Corps
Alan Shalloway – Creating a Model to Understand Product (and Software) Development
- 3 types of value: Discover what a customer needs, Discover how to build it, Build it
Jeff Patton – Lean Product Discovery
- There is a difference between Delivery & Discovery
- Undelivered software is a solution hypothesis (usually incorrect)
- Include discovery in the VSM
- Market demand = pull (but is post delivery)
- “Lets go to marble”
- Discovery Kanban + Delivery Kanban
- Visualisation creates collaboration
- Storyotype 1: Bare Necessity – minimally demo-able
- Storyotype 2: Capability + Flexibility – options-
- Storyotype 3: Safety – invisible
- Storyotype 4: Usability, Appeal, Performance – differentiating
- Chess analogy – Opening Game, Mid Game, End Game
- Use options when cost of iteration and failure is high
John Seddon – Re-thinking Lean Service
- If you manage costs, costs go up
- Failure demand is a signal
- Incentivising workers get less work done
- Predictable failure demand is preventable
- Lean is getting a bad brand
- The only plan is to get knowledge
- Improve flow – walk the flow
- Pull the help, keep the work
- Do “productivity” tools improve productivity?
- Make them curious
Tuesday
Don Reinertsen – Second Generation Lean Product Development: From Cargo Cult to Science
- Understand causal relationships and salient phenomena
- Scientific v Faith (Science Free) based approaches
- Faith based = Cargo Cult
- Every system has Push / Pull interfaces
- No bad tools, only wrong times to use them
- In Product Development, Requirements are a degree of freedom
- Inventory is information and invisible (physically and financially)
- Agile practitioners are going a better job at LPD than LM practitioners
- In engineering we are making economical (Profit + Loss) decisions and we have no clue what we are doing
- Quality = Process x People – If either drops to 0, the quality of the result will drop to 0
- Don’t tolerate initiative – demand it
- Chief Engineer doesn’t know better than the customer
Kenji Hiranabe – Learning Kaizen from Toyota
- Balance process control and process improvement
- Management fosters human potential
- Go to the gemba != PowerPoint
- To know and to understand are different
- Think within your constraint
Hal Macomber – Lessons from Target Value Design
- Meld planning with execution + control
- What signals do we pay attention to?
- Articulate + activate the network of commitment
- Only start work when it is in the condition to be finished.
- Embrace the contradictions of Lean
- Focus on tool users, not the tools
- PDSA – Study, don’t Check
- Creating constraints creates innovations
- Create constraints so that we can do our work
- Don’t open the trenches until you know you can fill them
Marc Baker – Lean Thinking: what is distinctive about it and where it is going?
- Pull means take the payment first
- Visual controls mean that nothing is hidden
- Standardised tasks are the foundation of employee autonomy
- Go see, ask why, show respect
XP As A Kanban System
Sep 25th
During recent discussions with XP folks on the topic of Kanban, it occurred to me that based on my understanding, XP can be described in terms of a Kanban System for Software Development. This is an attempt to do that, on the basis that it might be useful in helping teams understand Kanban concepts. I have structured the description around the five primary practices of Kanban that I have previously blogged about.
1. Mapping the Value Stream
Here’s an example of an XP Value stream
- A customer comes to the team with a problem they want solving, and collaboratively they write a set of User Stories
- The team then holds an Iteration Planning meeting to prioritise which User Stories they believe they can deliver within the next iteration (which is 2 weeks in this example)
- The team then pick the first of those User Stories, and plans how they will build it.
- They build a User Story (using good engineering practices)
- When the team has something to show, they review progress with the customer. In this case, every 2 days on average. I assume that the team will be collaborating with the customer more often than that, but am not showing that level of feedback for simplicity. Reviewing a User Story may result in re-planning and re-building (iterating). Completing a User Story will trigger another User Story being planned and built.
- At the end of the Iteration, the customer accepts all the completed User Stories. At this point the User Stories are re-prioritised again and another set chosen for the next Iteration.
- At the end of the Iteration, the software may also be released, and more User Stories may be written.
In practice, the workflow is not this precise, but I think its a close enough approximation. For example, new User Stories could be written at any point.
2. Visualising the Value Stream
An XP teams may use a very simple visualisation such as the one above, which focuses on the Plan-Build-Review section of the Value Stream. The User Stories (yellow) chosen for an Iteration are initially not started. When they are planned, then tasks (grey) are added, and the tasks are working on until they are all done, and the User Story is Done.
3. Limiting Work in Progress
An XP team will limit work in progress by working in pairs, with each pair only working on a single User Story at a time until it is Done. Thus in the above example, a team of four developers limits work in progress to two User Stories by two pairs.
4. Establishing a Cadence
XP teams have a very specific and synchronised cadence which is created by time-boxing the Iteration. Prioritisation, Reviews, Retrospection and Releases all occur at the same time interval, and User Stories are planned to be completed within that time interval.
5. Reducing the Kanban Tokens
An XP team is always striving to improve, usually by using retrospectives. As such, in the above example, they may reach a point where all four developers are able to work on the same User Story, and as a result complete it sooner.
Assuming that this description of an XP based Kanban System is not widely off the mark, I hope that it serves to communicate that XP and Kanban are not alternatives, but different and compatible ways of describing a process. Further, I have found that by understanding a process in terms of a Kanban System, it has helped my think about alternative ways of evolving that process in order to improve it. There is of course, more to XP than I have mentioned here, such as release planning, and the usual good engineering practices.
Balanced Software Development
Sep 16th
Agile2009 provided me with 3 sources of ideas which all complemented each other, and which I think make an important point that I want to repeat.
Firstly, on the flight over, I read John Shook’s blog post about his work with Starbucks. In it, he responds to the suggestion that by advising Starbucks on using Lean methods, he is transforming them into a robotic fast food joint like McDonalds. That suggestion sounds similarly like the common claim that Kanban transforms software development back into a robotic process. The piece that stood out for me was this:
Toyota combined old IE Scientific Management principles and techniques with social dimensions appropriate for the modern world. Even workers who do “manual labor” with their hands are knowledge workers. Front-line employees become the scientists.
By redefining roles, Toyota changed the answer to the question of who is the scientist in scientific management.
In other words, Scientific Management is still relevant for knowledge work, when the workers are the scientists. That keeps the balance between the Process and the People.
Secondly, Alistair Cockburn talked about three pillars of Effective Software Development in the 21st Century in his Agile2009 keynote. The 3 pillars are:
- Cooperative Game
- Craft
- Lean Processes
Again, this to me demonstrated the need for a balance between the People and Process focussed elements.
Finally, I attended Jon Dahl’s talk on Aristotle and the Art of Software Development. This focussed on the differing ethical philosophies of Kant, and Mill and Aristotle:
- Kant looked at the Acts, known as Deontology. This can be equated to looking at Process.
- Mill looked at the Effect, known as Utilitarianism. This can be equated to looking at Outcome.
- Aristotle looked a the Actor, known as Virtue. This can be equated to looking at the Participant.
Chatting with Matt Wynne after the talk, we both had the same thought. While individuals will probably sway to one form of these philosophies, there is room for all of them, and again, a balance is good. I would even go so far as mapping the three philosophies on to Alistair’s three pillars.
- Deontology is Lean Processes
- Utilitarianism is Craft
- Virtue is the Cooperative Game
In many of the recent discussion I have seen and been involved in on Lean and Agile, and Kanban in particular, it seems to me that most of the debate is because the various participants are talking from the perspective of one of these pillars or philosophies. We should remember that they are all important, and that achieving the right balance is what is going to help us be successful in delivering valuable software.
