Evolving a Workflow

Mary Poppendieck gave a talk on Workflow is Orthogonal to Schedule at Agile2009, during which she very neatly transitioned a schedule-focused view of work, into a flow-focussed view. At least I thought it was neat, so I’m going to try and reproduce the basic elements here, using my favourite agile workflow.

4 Week Time-box Schedule

workflow1

Here we have a schedule showing an average of 4 features being delivered every 4 week time-box. Each time-box is preceded by 2 weeks understanding the next features, and a release is made every 8 weeks. Its not a bad schedule. There’s a regular release every two months which is better than a lot of projects, but it could be better.

2 Week Time-box Schedule

workflow2

Now we reduced the time-box to 2 weeks, meaning that we do an average of 2 features in each time-box. This means that the preparation now takes only 1 week. Additionally, we are now releasing at the end of every time-box, which is also reduced to 1 week. Much better.

One Piece Flow

workflow3

If we continue the evolution we end up working on a single feature at a time and releasing it immediately. Each feature only takes a week to build, and preparation and release times are now down to 1/2 a week. At this point, we don’t really have a schedule anymore, but a natural workflow.

Cumulative Flow Diagrams

One of the things that strikes me about the diagrams above, is that each step in the evolution transforms them more into a smooth Cumulative Flow Diagram. In fact, in the same presentation, Mary showed the following picture taken from Building the Empire State. This is the building schedule from from around 1930, and itself looks remarkably like a CFD. Another example of how we are not inventing anything new, but can learn from other industries and successes.

empirestate

VN:F [1.9.22_1171]
Rating: 5.0/5 (1 vote cast)

The Fifth Primary Practice of Kanban

I recently wrote what I considered to be the four primary practices of a Kanban System for Software Development:

  1. Map the Value Stream
  2. Visualise the Value Stream
  3. Limit the Work in Progress
  4. Establish a Cadence

During subsequent discussions on the aspect of Continuous Improvement in a Kanban System, I decided that there was a missing fifth primary practice:

  1. Reduce the Kanban Tokens

I originally named this practice “Eliminate Kanban”, but was persuaded that this was probably overly sensational, and as a result potentially confusing or misleading. Its intent is that once a Kanban System is in place, the team should be constantly looking to improve it by creating an environment where the work flows naturally. There is a quote that I believe comes from Rother and Shook which says “flow where you can, pull where you must”. By striving to reduce the number of kanban tokens in the system, a team will move towards an environment where they are more self organising and the work can flow. This can be achieved by either lowering the WIP limits or by collapsing the number of distinct stages.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

Isn't Kanban Just a Task-board?

While the word Kanban comes from the Japanese for “visual card”, the term Kanban as used by the Kanban Software Development community, represents much more than a standard task-board, or team-board. Additionally, the Kanban Software Development community have not tried to replicate the mechanism of the Toyota Production System kanban tool exactly, but have taken the underlying principles in order to achieve similar effects in software development. So what is a Kanban System for Software Development?

A Kanban System visualises some unit of value. This unit of value could be a User Story, Minimal Marketable Feature, Plain Old Requirement or something else. This is different from a task-board, which generally focuses on visualising the current tasks.

A Kanban System manages the flow of these units of value, through the use of Work In Process limits. This is different from a task-board, which generally has no WIP limits, but aims to have all tasks complete by the end of a time-box.

A Kanban System deals with these units of value through the whole system, from when they enter a teams control, until when they leave it. This is different from a task-board, which generally only deals with the work in the build/test stage, but shows no information about what work is being prepared, or what work is ready for release.

By putting these 3 properties of a Kanban System together, we can describe a Kanban System for Software Development as one which allows value to flow through the whole system using WIP limits to create a sustainable pipeline of work. Further, the WIP Limits provide a mechanism for the Kanban System to demonstrate when there is capacity for new work to be added, thereby creating a Pull System. Finally, the WIP Limits can be adjusted and their effect measured as the Kanban System is continuously improved.

A task-board simply shows what development tasks have been predicted to be done in the current time-box, with their status.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

Anxiety or Boredom Driven Process Improvement?

At the SPA conference recently, Joseph Pelrine talked about “Flow: The Psychology of Optimal Experience” by Mihalyi Czikszentmihalyi. The ideas behind this struck a chord with me as a way of describing something I originally said when discussing whether kanban is only suitable for mature teams. That is that rather than focusing on being Agile which may (and should) lead to being successful, Kanban focuses on becoming successful, which may lead to being Agile.

Mihalyi Czikszentmihalyi describes the state of Flow as having a balance between ability, and the skills required for a piece of work. If some work requires more skill than a person has ability, then they are in a state of Anxiety. If a person has more ability than the skills required for a some work then they are in a state of Boredom. Applying this to Process Improvement, we want to move teams up the Flow Zone so that skills required and ability increase equally.

Flow1

Increasing Skills and Ability equally at the same time is unlikely, so in practice there are two routes to move up the Flow Zone. The first is what I am dubbing “Anxiety Driven Process Improvement”. Move a team into a state of Anxiety such that they need to improve their skills to cope. I assert that this is the approach that makes time-box based method so effective. Time-boxing forces teams into a place where they need to improve their skills in order to deliver working software every few weeks. Many teams push back, and a common approach to this is to make the time-box shorter to emphasis the point! The other route is to do “Boredom Driven Process Improvement”.  Highlight to the teams where they need to improve their ability, and allow and support them in doing so such that they are able to taken on work requiring more skill. I further assert that this is the approach that kanban systems take. Visualising queues and work in progress in order expose the bottleneck where the ability needs improving.

Flow2

This might sound like I am suggesting the time-boxing and kanban are mutually exclusive approaches. That isn’t the case – I have already said that! Instead, I just find it an interesting angle to look at a different dynamic between two approaches.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

The Kanban, Flow and Cadence Simulation

I use a simulation when I have time in presentations on Kanban, Flow and Cadence, which I hope makes the ideas a bitter clearer and more concrete.  I was asked if I could make the materials available, so here they are with an explanation.  Its not perfect, and can be slow and time consuming, so I’d love any feedback on how to speed it up and make it a bit snapper.

The simulation uses an example work flow with 5 stages; Analysis, Design, Code, Test and Release, and a set of Minimal Marketable Features which require varying amounts of effort at each stage.  The kanban board is set-up as below with WIP and Completed columns for each stage

kanban-board
The MMFs can be printed from this MMF document.  If you want to create your own, they are generated from a MMF spreadsheet with a MMF mail merge.

An MMF progresses through the system by rolling a dice for each days work per stage and subtracting the appropriate effort from the relevant total.  When a total reaches zero, the MMF can progress to the next stage.

mmf-card

This MMF requires 3 units of Analysis effort, 1 unit of Design effort, 4 units of Code effort etc.  If the Analysis role throws a 2, then there is 1 unit of effort remaining.  If they throw a 3 then there is 0 effort remaining and the MMF can proceed into Design.  If they throw a 4, then there is 0 effort remaining, the MMF can proceed into Design, and 1 unit can be used on the next MMF.

The simulation proceeds by each role throwing a dice once per day, from Analysis to Release, and pushing work through the system.  After each week (5 days) we can update the metrics spreadsheet with the current status.  For each MMF, set its its status in the Column for week 2 (i.e. the beginning of week 2).  In addition, to show how much work was started each week, set the status for all the new MMFs to Not Started in the Columns for week 1.

metrics-week-2

Continue for 3 or 4 weeks and you should start seeing a bottleneck.  The effort numbers on the MMF cards are deliberately weighted so that Code requires more effort than the other stages.  Hence in the CFD you can see that Test and Release are starved, while and Design is backing up.  You can also see that Lead Time is going up, but that Throughput has settled at 4 MMFs per week.

cfd-week-5 lead-time-week-5 throughput-week-5

If we continued to work this way we can predict that these trends would continue.  Code would be a bottleneck, causing lead time to grow, and constraining throughput.  However, at this point we can introduce kanban limits. I tend to set a single limit across both the stage and its completed column as I find this simpler.  An alternative could be to have each column have a limit of 2.  As we are implementing a kanban system we should also pull the work by rolling the dice from Release back to Analysis.

We can also re-focus our available effort to help Code be more productive.  We do this by artificially weighting the dice.  A normal dice has an average throw of 3.5.  By reducing the average throw to 3 for Analysis and Test, we can increase the average throw for Code to 4.  The following table shows how this works.  For example, to get an average throw of 3, a roll of 5 or 6 translates to 4.

dice-weighting

Running like this for another 3 or 4 weeks should clear the backlog, giving smoother flow and resulting in Lead Time coming back down.  Throughput may also improve  as we have made Code more productive.  At this point further tweaks to the system may suggest themselves.  Shifting the allocation of effort by further weighting the dice may is one option.  Adjusting the limits is another.  Run until all the MMFs have been completed and see how the graphs look.

cfd-week-13lead-time-week-13throughput-week-13

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

Kanban, Flow and Cadence

Intro

There has been some noticeable increase in interest in Kanban recently, with a number of people asking for more basic info, and more people writing new blogs and articles.  This is my attempt to describe in more detail my take on it all, which I refer to as Kanban, Flow and Cadence.

  • Kanban – Controlled Work
  • Flow – Effective Work
  • Cadence – Reliable Work

Kanban

A Kanban system is a mechanism for controlling the work in the software development system.  Kanban can be defined as “visual card”, as shown below – kindly written for me by Kenji Hiranabe at Agile 2008.

The origin of kanban is the Toyota Production System. Taiichi Ohno, in his book Toyota Production System, wrote:

“The two pillars of the Toyota production system are just-in-time and automation with a human touch, or autonomation. The tool used to operate the system is kanban.”

Kanban is what enables a pull system for just-in-time work.

What does a Kanban System look like for software development?  Very simply, there is a queue of work, which goes through a number of stages of development until its done.  When work is completed in a stage, it goes into a downstream queue for the next stage.  When someone needs new work to do, they pull it from their upstream queue.  This can be depicted as below.

That looks very like a typical Agile Task Board, with maybe a few more steps, although there is nothing to say there can’t be a single development stage. However, there is one more important element which really defines a kanban system – limits.  There are two basic limits – Queue limits and WIP limits.

Queue limits are designed to avoid premature work.  This how just-in-time is achieved.  The limit should be large enough to keep the team busy (i.e. there is always something in it for the team to start work on), but small enough to avoid premature prioritisation (i.e. having things sitting in the queue for too long before they are begun).  Ideally the queue should be FIFO, although this is a guideline rather than a hard rule, as sometimes available skills or other resources mean that it is not always possible.

Work In Progress limits are designed to reduce multi-tasking, maximise throughput, and enhance teamwork.

Reducing multitasking is beneficial for two primary reasons.

1) 20% time is lost to context switching per ‘task’, so fewer tasks means less time lost (from Gerald Weinberg, Quality Software Management: Systems Thinking)

2) Performing tasks sequentially yields results sooner.  As the diagram below shows, multi-tasking A, B and C (on the top), delivers A much later, and even C slightly later, than sequentially (on the bottom).

A great exercise to demonstrate the effects of multi-tasking was described by Clarke Ching.

Throughput is also maximised by decreasing WIP.  Simple examples of this effect are traffic jams, where traffic speed reduces as more traffic builds up, and CPU load, where application performance goes down as CPU load increases.  The effect can be be explained by looking at Little’s Law for Queuing Theory:

Total Cycle Time = Number of Things in Progress / Average Completion Rate

Therefore, to improve cycle time there are two options; reduce the number of things in process or improve the average completion rate.  Of the two, reducing the number of things in progress is the easier, and once that is under control, then the more challenging changes to improve completion rate can be applied.

Finally, by having fewer work items in progress, then the team is able to focus more on the larger goals, and less on individual tasks, thus encouraging a swarming effect, and enhancing teamwork.

Limiting Work In Progress like this can seem unusual for teams, and there is often a worry that team members will be idle because they having no work to do, but are unable to pull any new work.  The following guidelines can be useful to help in this situation.

  1. Can you help progress an existing kanban? Work on that.
  2. Don’t have the right skills? Find the bottleneck and work to release it.
  3. Don’t have the right skills? Pull in work from the queue.
  4. Can’t start anything in the queue? Is there any lower priority to start investigating?
  5. There is nothing lower priority? Find other interesting work.

They key question here are what constitutes lower priority investigative work or other interesting work.  Essentially it is work which won’t create any work downstream, will improve future throughput and can be paused as soon as existing kanban related work is available.  Lower priority work could be spikes or analysis for known impending work.  Other interesting work could be refactoring, tool automation, personal development or innovation.

WIP limit sizes can depend on type of work and size of team and should be adjusted to achieve maximum flow.  One approach is to start small (e.g. a limit of 1) and increase as necessary.  Another is to start larger (e.g. a limit of half the team size) and reduce until the sweetspot is achieved.

The consequences of using a kanban system are that the Product Backlog can be eliminated, because the immediate queue is the only work of interest, timeboxed iterations (i.e.Sprints) can be eliminated, because work is pulled as necessary, and estimation can be eliminated, because work is not planned into iterations.

Flow

Flow describes how the work in the system can delivery maximum value.  As Mary and Tom Poppendieck write,

“In lean enterprises, traditional organizational structures give way to new team-oriented organizations which are centred on the flow of value, not on functional expertise.”

In particular, Lean emphasises ‘One Piece Flow’.  This means moving one piece at a time between stages in a workflow as opposed to moving  batches of work between stages in a workflow.  The ‘One Piece’ in a Kanban system for software development can be thought as the Minimal Marketable Feature, as described by M Denne & H Cleland-Huang in Software by Numbers.

“A minimal marketable feature is a chunk of functionality that delivers a subset of the customer’s  requirements, and that is capable of returning value to the customer when released as an independent entity”.

The kanbans should be minimal so that they are as small as possible in order to enable progressive delivery  to realise the product sooner, reduce feature bloat and focus on the the core features which are the most important, and minimise complexity because each feature has a cost to a user.

The kanbans should be marketable in a number of ways.

  • Table Stakes – these delivery parity to the competition and are the minimum needed to be in the game
  • Differentiators – these differentiate the product from the competition and delight the user
  • Spoilers – these nullify a competitors differentiator and raise the bar for parity
  • Cost Reducers – these reduce cost and improves the profit margin

A useful guideline is that an MMF is marketable if it is something that could be written about on a product blog.

The kanbans should be features which are distinct, deliverable and observable.  The INVEST acronym (Independent, Negotiable, Valuable, Estimable, Small, Testable) as described by Bill Wake, can also be useful for applying to MMFs.

The Marketable element of MMFs means that they may sometimes be larger than typical User Stories, which often break work down such that while they can be incrementally delivered, and show some element of value, they are not marketable in their own right.  Therefore, it is important to understand an MMFs Value Stream in order to deliver the whole MMF as quickly as possible.  A value stream describes the steps, delays and information required to deliver a product, and can often be used to decide the steps in an initial kanban system.  With large MMFs, the User Stories become more of an analysis technique in order to enable incremental delivery of an MMF, without losing sight of the overarching MMF.  I describe how a continuous flow can be achieved with MMFs in The Anatomy of an MMF.

A number of techniques can help manage the relationships between MMFs and User Stories in order to realise the benefits of both.  One is User Story Mapping, as descried by Jeff Patton.

I have also recently been working in a regulated environment where User Case Goals and Sub Goals have provided the MMFs, with the detailed scenario paths and steps providing the additional details.

A further enhancement is to use two-tier Kanbans, with one tier for the MMFs, and another for the User Stories.

The consequence of applying the concept of Flow is that emphasis is placed on using larger, value-focussed MMFs, rather than smaller, more incremental Stories.

Cadence

Cadence is the approach to achieving commitment and reliability with a kanban system.  I often get a question something along the lines of,

“If the team isn’t estimating or planning with fixed time-boxes, how can it make reliable commitments?”

To quote Mary and Tom Poppendieck again,

“A regular cadence, or ‘heartbeat,’ establishes the capability of a team to reliably deliver working software at a dependable velocity.  An organization that delivers at a regular cadence has established its process capability and can easily measure its capacity.”

The time-boxed iteration is one form of cadence, which couples planning, review and release together.  A kanban system de-couples these events and allows them to have separate cadences, as well as adding two additional ones.  Throughput is the amount of output of a process in a given period of time, and Cycle Time is the length of time to complete a process.  The relationship between the two is:

Throughput = WIP / Cycle Time

Throughput allows forecasting of future capability, without needing to specify what might be delivered.  Cycle Time allows commitment by becoming an SLA with the business (see Kanban Commitment).  Where the size of work varies, from large new features to small fixes and change requests, then a classification of MMFs can enable a range of cycle-times.  Both Throughput and Cycle Time can be charted and trended, in a similar way to XP’s Velocity, as a means to encourage the team to improve.  A Cumulative Flow Diagram can also make visible how the work is flowing through the system and highlight bottlenecks.

For longer term forecasting, a quarterly planning cadence focusses on quarterly goals and objectives.  MMFs can subsequently prioritised to meet those goals and objectives.  A regular release cadence will build up trust that the team will work to its full capacity and throughput.

Other cadences, are daily stand-up meetings, and regular retrospectives and Operations Reviews as described by David Anderson.  Some teams are using a Retrospective Kanban to signal when a retrospective is necessary, and I have already blogged briefly about Kanban and Retrospectives.

The consequence of Cadence is that commitment and reliability is achieved though measurement as opposed to planning.

Summary

They key points of Kanban, Flow and Cadence are:

  • A Kanban System manages the workflow in a way which allows the Product Backlog, Timeboxed Iterations and Estimations to be eliminated.
  • Flow is about effectively delivering maximum value by focussing on optimising the value stream of larger MMFs
  • Cadence allows iteration input and output to be de-coupled and achieves commitment and reliability via measurement rather than planning.

Further resources and information can be  on my Kanban Page, including most of the material which has influenced and directed my thinking.

VN:F [1.9.22_1171]
Rating: 4.8/5 (29 votes cast)

The Anatomy of an MMF

I’ve been involved in a number of discussions about how User Experience work fits into an Agile process.  As a result a trying to articulate my position, I’ve come up with the following explanation of the anatomy of an MMF.

The following diagrams assume the following key:

Lets start by looking at a typical waterfall model:

In this case, all the UE work would happen early on, but obviously, isn’t Agile, so doesn’t allow for iteration and improvement.

A more Agile approach is:

In this case, the UE work is spread across the lifetime of the project, but for each increment, UE work happens early, relative to build etc.

That’s still not truly representative of an Agile project, which is more collaborative, so we have:

Here, everything happens together, as needed, for each Product Backlog Item (PBI, using Scrum terminology) and this is where the problem begins.  In order for a PBI to be sufficiently formed to be able to be planned into a Sprint, there may need to be Analysis and UE work in advance.  So typically these become PBIs of their own in preceding Sprints.  The problem I have found with this approach is that the focus is on the PBI, and the over-arching feature of value is lost, resulting on long cycle times.

So this is how I prefer to think about it:

Throughout the life of a Minimal Marketable Feature, the different disciplines are involved to varying degrees.  More UE earlier on, and more QA later.  However, the focus is still on the over arching feature, and all are still collaborating to deliver the feature as quickly as possible.  Any Sprints or PBIs are simply mechanisms to manage the flow.

Given this view of things, its a small step to allow MMFs to overlap, so that the features can smoothly flow through the system.

VN:F [1.9.22_1171]
Rating: 4.0/5 (1 vote cast)

KFC Consequences

One of the regular questions I get asked when I talk to people about KFC Development is about how it is different.  As a result I came up with a set of KFC Consequences to try and help articulate this.

  • Kanban Consequence – Eliminate backlogs, timeboxed iterations and estimates.

Eliminate might be a slightly strong word, but there is certainly less reliance on these things, and thus less wasted investment in them.  Ultimately a high performing team would not need them in a typically recognisable form, although they may use variations.  For example, an ‘incubation area’ might replace the backlog as a simple place to keeping ideas before they are refined into MMFs.  Similarly, classification might replace estimation.

  • Flow Consequence – Use larger, value focussed features

Rather than an emphasis on driving user stories into smaller and smaller pieces, there is more of an emphasis on right-sizing pieces of work based on their value.  User story decomposition is used more as an analysis technique.

  • Cadence Consequence – Commitment via measurement and SLAs, rather than planning and timeboxes.

Instead of planning batches of work into timeboxed iterations as a mechanism drive productivity and commitment, the mean lead time of MMFs is measured and used as an SLA against which due date performance is reported.  Thus it is actual performance which drives productivity, rather than forecast performance.

VN:F [1.9.22_1171]
Rating: 0.0/5 (0 votes cast)

KFC Development

I’ve been referring to my latest thinking on development process as KFC Development. Its a bit of a gimmicky name, which makes it memorable, but there is some meaning behind it. KFC stands for Kanban, Flow and Cadence – three lean concepts which I think are important and complementary.

Kanban – Kanban is the mechanism which controls the workflow and helps manage inventory and investment, and identify bottlenecks.

Flow – More specifically One Piece Flow. The one-piece is a Minimal Marketable Feature (MMF) which ensures there is focus on the delivery of actual value. This avoids the temptation to artificially break work down into smaller chunks in order to fit them it into an iteration or sprint. A side effect of this flow, is that typical agile time-boxed iterations can become unnecessary.

Cadence – Cadences give the team the rhythm usually provided by time-boxed iterations. There can be a variety of cadences, from daily stand-ups, to quarterly roadmap planning. An additional cadence is that defined by cycle-time, which determines throughput, and allows a level of forecasting of future work.

I’m hoping to explore these ideas more in the future.

VN:F [1.9.22_1171]
Rating: 5.0/5 (2 votes cast)