What is Agile Product Development?

Agile product development is the ability to impact the market, utilize this impact and to have the ability to make use of new opportunities appearing in the marketplace.

In this article I will define agile product development. I will start with a definition of product development, followed by a short history of the term agile. Finally I’ll describe how the different strands of development of agile converge in Pulse.

The Definition of Product Development

Product development is the transformation of an opportunity on the marketplace into a product in the client’s hands. The opportunity may have been created by the company through analysis, campaigns, technology development, collaboration or other measures. A company can also make use of an opportunity that has appeared on the marketplace. Product development presupposes the ability to (1) create business models, (2) establish cooperation between clients, suppliers, distributors as well as other stakeholders, and (3) develop and nurture products and production systems.

The History of Agile – War

Agile/agility is a term that came into use among military researchers at the end of the 1970’s to describe the need for more agile military operations. A deciding factor of success in military operations is the ability to acquire information about current events, create an understanding of the situation, choose a plan of action and to implement that plan.

The first military strategist to discuss agility was the Prussian general Carl von Clausewitz in the beginning of the 19th century. After a life-long career in the military he drew the conclusion that a battle contained so much uncertainty that the plans quickly turned obsolete once faced with reality. His solution was quite unconventional: instead of creating even more intricate plans (which many still presume to do today), he told the soldiers what the ultimate goal was. Clausewitz’s idea was to let the soldiers solve the challenges as they appeared. It was an idea that a lot of people had trouble accepting. Despite this opposition, this agile philosophy was partially used in the Franco-Prussian War (1870-1871) with great success.

By World War I the Germans had forgotten about this success and once more relied on top-down detailed management with well-known results. After World War I the Germans introduced what came to be called mission command (auftragstaktik) and control freedom (truppenführung). When these were implemented in the German army they realized the importance of keeping the generals at the front –  not to micromanage but to better put what happened on the battlefield in a strategic context. The German army was initially very successful (1939-1944). However, when the war dragged on the British could make use of their agile abilities on a political level. Nazi Germany never understood the purpose of democracy. Their political system was based on strong leaders with overlapping responsibilities and where information was power. In the democratic United Kingdom the power came from the work involved in acquiring and sharing information.

From the military we can learn three things. 1) There is great insecurity in detailed plans  and control needs to be given in small steps as it happens. This step-by-step planning and control is preferably done by the people who have to perform the task. 2) There is a need for a “general” at the front who can put events at an operative level in a strategic context. Overall strategic plans must be able to be adjusted as the situation changes. 3) Superiority in the battlefield is not enough to win a war. One must also be superior at a political level within the highest leadership.

The History of Agile – Car Manufacturing and Lean

The fact that plans become obsolete the moment they face reality is something that Taiichi Ohno, the product engineer of a car company, realized in the 1950’s. His solution was to replace top-down planning with day-to-day control out in the production line. He developed a method that made it possible for the operator in the production line to order more materials as they were being used. This method is known as just-in-time. By working in this manner he decreased the uncertainty involved in planning and management. With just-in-time there is considerable less material in circulation and faults and flaws become more visible. He complimented the just-in-time principle with jidoka, a method to take control of the problem at the source. With these methods, the operators were given the mandate to identify and solve problems themselves. Just-in-time and jidoka created an almost magical increase in both productivity and quality, which were consequences of reduced uncertainty. Much later on these principles came to be referred to as lean.

Visuell styrning sänker osäkerheten.
Visual management reduces uncertainty, which leads to increased productivity.

From lean we can learn that day-to-day management, just-in-time, reduces uncertainty when it comes to planning. We can also learn that management and problem solving at the source improve a company’s ability to make decisions.

The History of Agile – IT Projects

In the 1980’s and 1990’s many companies introduced decidedly more bureaucratic models for product development. These models led to a decline in productivity and quality. As early as the 1960’s, researchers demonstrated that bureaucratic models lead to stagnation. Eventually there was a counter-reaction against the bureaucratic models, particularly among the developers themselves. In the 1990’s Sutherland and Schwaber produced a development model that utilized short planning cycles. They called their model Scrum. Within Scrum the project manager was removed and instead it’s the team that is in charge of planning and solving problems as they come up. A decisive role in this model is that of the product owner, who has a responsibility to put the project work in a strategic context. In Scrum there are few roles and the model is so simple that it can be explained using a few sentences.

Scrum was inspired by research. One example of this is the article The New New Product Development Game in which the rugby term Scrum is used. Scrum is one of the few models that conforms to the Agile Manifesto which was first published in 2001.

From Scrum we can learn lessons about reduced uncertainty through iterative planning, teamwork, how just a few roles are necessary and the importance of an increased role for the product owner.

The History of Agile – Research in Complexity

Wikimol. http://commons.wikimedia.org/wiki/File:Lorenz_attractor_yb.svg. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.
Wikimol. http://commons.wikimedia.org/wiki/File:Lorenz_attractor_yb.svg.
This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.

Complexity research has helped organization theory explain basic phenomenon. From Edward Lorenz we’ve learned that in complex systems only short-term plans or forecasts are possible. Lorenz was a meteorologist and studied weather patterns. He discovered that weather forecasts are impossible to make for periods longer than a few days ahead. It’s the same mechanics that make it impossible to plan projects in detail. Complexity research also shows us how normal abnormal events are. Each abnormal event by itself is rare but put together they form a major part of the managing product development. This is what causes development models and process models such as PROPS to create more problems than they solve.

Complexity research also show that groups in which people work together and share information have a superior ability to solve complex problems. A team can generate more versatility and more inner complexity, which is necessary for understanding increased complexity in the surrounding world. This explains why mission tactics, teamwork, and other decentralized working methods are better than top-down efforts such as order management and target management.

The History of Agile Product Development – Pulse

Pulsrummet där människor möts i olika grupper.
The Pulse room, where people meet in different groups.

Scrum is a model for managing projects. Pulse is a model for managing an organization. At Pulse we’ve built on earlier, pragmatically developed methods. We’ve advanced the development of these methods and complemented them with new methods based on research in complexity. Pulse is based on day-to-day management, just-in-time. We work in teams in which information is shared through work. The shape of the company’s policy and direction is formed dynamically by company management. Strategies are formed with the aim of creating an impact on the market, and based on the possibility of new opportunities. This happens through a network of daily “pulse meetings.” The strategies are implemented in the form of projects related to, among others, marketing and R&D. We have few roles, namely vice-president, marketing director, development director, program director, mission director and Pulse project manager. The program director is the equivalent of a product owner within Scrum and general in mission command.

At Pulse, the network of Pulse meetings determine the organizational structure. Information is shared at Pulse meetings, in workshops and at demonstrations. By using this network of organization, Pulse builds agility. Pulse is a model for agile product development.

Sociotechnology and Lean Have a Lot in Common

Visuell styrning sänker osäkerheten.

Lean is a method of working that relies on decentralized and self-organized teams. Sociotechnology and lean are closely related.

A lot of criticism has been levied at lean for causing stress and contributing to a negative working environment. Some Swedish researchers link lean with a reborn form of Taylorism. One reason for this criticism is that a lot of what is marketed under the label lean in reality comes from Taylorism and bureaucratic traditions. The result of this is documented in the book Lean i Arbetslivet (translation: Lean At Work).

Lean is Daily Management and Decentralization

As I wrote in previous entries, lean consists of two principles: visual management and “deal with problems as they appear” (fault tolerance).

Visual governance means the use of visual signals to coordinate work rather than forecasts and plans. These signals create a real-time management that is based on the current situation and plan for the next couple of hours. There is considerable less uncertainty in visual governance when compared to traditional large-scale planning. Less uncertainty at work means less interruptions, which leads to higher productivity.

Fault tolerance refers to the implementation of mechanisms to catch and deal with problems and troubles that appear without pausing operations. The problems should be taken care of by the people closest to them. In production the people closest are the operators and within projects it’s project participants. When problem-solving is distributed within the organization, decision-making is given greater “bandwidth” regardless of whether decision-making and problem-solving is done by management, project leaders or experts.

Visual governance and fault tolerance are principles that need to be converted into appropriate methods that suit the organization with implemented together with lean.

Misconceptions About Lean

There are many misconceptions about lean and one reason for this is because the uncertainty that’s present in long-term forecasts only becomes apparent many years later. One common misconception is that lean requires the invention of standardized processes in production and constant improvements. However, this has been known since the 18th century. Some people also believe that standardized processes are vital for lean to work but that isn’t true. On the contrary, a rigid thought process in a social organization creates large problems, as demonstrated by sociotechnology research.

Sociotechnology

Sociotechnology was born at the Tavlstock Institute following World War II. In sociotechnological system theory, the organization is divided into two systems: technical and social. The technical system consists of equipment, machines, tools and methods. The social system consists of people, interactions and knowledge. The technical structure affects the social organization.

Concept for a sociotechnical system

Research conducted by the Tavlstock Institute demonstrated the negative consequences of Taylorist solutions, including problems with unclear work duties, individually designed tasks, top-down management, constricted thinking with clearly defined roles and standardized working methods.

Sociotechnology proposes self-organized teams where tasks and responsibility are determined by the work that is accomplished. Research shows that sociotechnology leads to higher productivity compared to Taylorist solutions and the explanation for this is found within the human equation: people working in teams were more motivated.

Today, thanks to complexity theory and systemized thinking, we have a slightly different explanation. Taylorist solutions don’t lead to high productivity as was thought; on the contrary they lead to inefficiency. The reason for this is that in a social system, standardized processes with little room for freedom lead to chaos. In order to create order in whole systems and keep up high productivity, a large degree of freedom at the individual level is necessary. That is one needs a multidimensional system for both management and workers.

This means that a certain lack of order at the individual level leads to order within the organization as a whole. This is far from intuitive but the results of complexity theory are convincing and backed up by empirical organization research.

The problem with the sociotechnological system school is the lack of suitable methods. This is where lean enters the picture.

Implementation of Sociotechnology and Lean

A correct implementation of Lean can be characterized by the following:

    • Duties are broken down into manageable sections since that facilitates the distribution of the workload. In Taylorism, duties are separated.
    • With Pulse, we use manageable sections with larger, related duties.
    • Work is done in groups, teams and projects to complete the sections. In Taylorism, work is done individually.
    • Within FoE we work in projects and teams. At Pulse these are self-organized.
    • That the people who work in groups, teams and projects are encouraged to be able to take care of several different duties since that makes it easier to distribute work and problem-solving. Taylorism requires maximum specialization.
    • With Pulse, there are few roles. Cooperation is the key. No activities are dedicated to a specific role or position. People choose what to work on.
    • There are few permanent roles. In Talyorism, the defining of different roles is very important. Within Pulse, there are just a few permanent roles.
    • Increase individuals’ and groups’ ability to act so that everyone can adjust their actions to the current situation and unforeseen events. The goal of Taylorism is to decrease the ability to act through standardization and optimization of processes.
    • With Pulse there is no standard way to work or process thinking. Pulse is a part of the technological structure and it is within this structure that visual management is used.

The above list, which details the components necessary for a successful implementation of Lean stands in contrast to Taylorism and bureaucracy. Instead, it aligns with sociotechnical systems. When correctly implemented, lean is a way to realize the ideas behind sociotechnology.

Socioteknik och lean - produktion
Within production, the system is shaped to facilitate teamwork. Visual management and distributed problem-solving are utilized.

 

In production, the kanban is used for visual management within the technical structure.

Socioteknik och lean - FoU
Within strategy and development a network of Pulse meetings are used to facilitate teamwork. The Pulse meetings use visual management.

Within strategy and development, the physical spaces in the form of the team room, Pulse room and Pulse boards make up the technological structure. The Pulse meetings form a network that we call an agile network organization.

Sociotechnology was created in the 1960’s and is based on the knowledge available then. The past decades have seen a fast development within organizational theory based around complexity research. The main tenants of sociotechnology, including self-organized teams, have strengthened in recent years while the explanation models have changed considerably.

Today there are methods to implement sociotechnology with the help of Pulse.

Obeya – 20 Years Old

An Obeya is a large room for visual management and jidoka. Obeya was invented in 1993 as part of the Prius project with the aim to shorten lead times.

I first heard about Obeya and periodic stand-up meetings in 1998 while working with operation development at Scania. What I heard about Obeya had so many surprises that it really took me a while grasp. At this point in time I had, from time to time, been involved in different efforts to increase the productivity in product development. Until then, the development engineers had always been the focus of our work. We tried to increase productivity with the help of planning and different project models. Obeya changed our ideas of how problems should be solved. Not only do you meet standing up at an Obeya but Obeya meetings are considerably shorter –  minutes instead of hours – compared to normal meetings. However, the big surprise for me was who participated in the Obeya meetings: management.

Clues to the Secret of the Obeya

It took me several years to understand what made these visual meetings with management so powerful. I found clues in books written by organization theorists (such Herbert Simon and James G. March). Further clues came during the development of The Pulse guide in the early 2000s. During this time Ulla and I created a model with a Pulse room that suited the organization and culture of Nordic technology companies.

In a Pulse room information and feedback is visualized so that everyone can see what’s going on. Besides frequent meetings between designers that work with projects and tasks there are also regular meetings with management. These managers represent different areas such as construction, acquisitions, production technology and marketing. Joint decisions are made and mutual action is taken. This creates heavy leveraging at the top of the operation, increasing the organization’s ability to take appropriate actions. Learning increases.

Obeya was invented between 1993-1994 when Toyota created the development project for the Prius hybrid. To develop a new car model is normally an incredibly complex task. However, with Prius they also had to develop a new driveline. These kind of projects can take a lot of time with lead times reaching up to ten years. The goal of the Prius was to launch the car in 1997 which was seen at the time as “mission impossible.” Against all odds, Toyota reached its goal and the car was developed in just three years. With this success Toyota didn’t just have a new product to offer the market but a whole new organizational model! This model might have been a more important success than the hybrid.

Obeya and Prius

The Obeya used during the Prius project was in many ways a kind of “project room.” We refer to this kind of project room as a Pulse room. In the Pulse room multiple projects can be developed. In the Pulse room the whole operation’s strategy and development work take place. All levels and functions are included in the Pulse guide and by doing so we could replace traditional project management with visual management. Furthermore, we could let the project’s Pulse board become the information and feedback that management use in order to help with decision-making. We also supplemented the concept with the organization’s strategic development. This meant that management and specialists that developed new markets, new products and new production technology solutions became a part of Pulse.
Anyone who has tried an Obeya or a Pulse room with management meetings has probably realized one thing: it’s a big success. However, there are pitfalls that can cancel out part of the positive effects. In some cases I’ve seen a bureaucratic viewpoint visualized when the management use visualization to control and monitor the project workers. In the bureaucratic tradition it’s also common for management, probably subconsciously, to have a personal agenda. When correctly implemented, management needs to place the problems and progress achieved within the strategic context of the operation. There is also a need to work together with the use of visual decision-making meetings; this type of coordinated action leads to success.

In an article in Ny Teknik, a Swedish technology publication, you can read about how Toyota has now created a new development center based around the Obeya concept. This is a strong  indication of how important Obeya is for Toyota.

What is Lean?

What is lean? I would like to answer that question with an explanation based on the description given by the inventor of lean, Taiichi Ohno.

Pull and push

Hopp and Spearman define pull in their work Factory Physics based on Little’s law. It’s a good definition that states that pull sets limits on any current workload while push doesn’t. However, there’s more to lean than pull.

What is Lean?

The inventor of the concept of lean is Taiichi Ohno. Ohno has written two books in which he summarizes the results of decades of experiments. These experiments and results are both comprehensive and have been tested by others. Ohno condenses the results into two principles: Just-In-Time (JIT) and jidoka (autonomation). Just-In-Time is the result that Ohno would like to achieve and the solution is Kanban (visual management). Kanban makes it possible for different actors to work together without the intervention of management or other planners. Besides these principles, Ohno also discusses 5S and wastefulness; however, his main focus is the challenge of getting people to work together.

Lean is Self-Organization

People who interact without the interference of outside actors practice self-organization. The term was defined in complexity research and is based on Claude Shannon’s information theory. When a number of actors, such as the participants of a group, have more interactions (more organization), the probability collaborating behavior (value-creating work) increases. This is called self-organization. An important prerequisite for self-organization is that the interactions are initiated by the actors themselves. There can’t be any outside actors (bosses, project leaders, etc.) that hinder or block interaction. However, the group does need information and feedback in order to deal with events appropriately.

Within a company, individuals will be a part of a greater whole (an infrastructure) that sets prerequisites for the groups to function well. In production, the production facility is this infrastructure. Within strategy and development we build an infrastructure through a network of Pulse boards and Pulse meetings.

Lean is Networks

With self-organization, the group has an opportunity to act more coordinated. Work that has previously been waiting for the contribution of another actor can now be done. However, with increased action there are also more problems that need to be solved in order for work to progress. The probability of problems appearing increases the greater organized the organization is. Therefore there is both a positive and a negative side. We can turn this into an equation:

Self-organizing (the ability to do things) = the degree of organization - problems that appear

With the help of visual management (such as Kanban, Pulse or Scum), increased organization is made possible. With the help of Jidoka (“pull the string” and other “take-care-of-problems-now methods”) you learn to deal with the problems that come up. When the “ability to do things” increases it’s a measure of increased productivity and quality. Productivity and quality improvements emerge from the interactions that come from self-organization. In theory, lean is therefore a concept that makes self-organization possible.

From Large-Scale to Networks

The time before lean was dominated by large-scale thinking. Nowadays we think in terms of networks. This has changed the duties of management. Previously, management lead and distributed work while in today’s world they build networks.

 

Read a blog entry about lean in action here.

Visual Management

Visuell styrning sänker osäkerheten.A lot of people who have worked with projects have experienced the same thing: downtime. You have to wait for people to get back to you, wait for results and wait for decisions to be made. On top of that other actors are waiting for you to finish your task that you can’t finish because you’re waiting for something or someone.

Forecasts are made to coordinate work in a businesses and projects. These forecasts tend to be wrong, which has led to the acquisition of more advanced, and expensive, forecasting tools, often in the form of some sort of IT solution. However, the forecasts are almost always wrong and thus we wait.

When you make detailed plans you make a future forecast. It’s just as easy or hard to forecast what will happen in four weeks within a project as it is to forecast the weather at the same time. In practice it’s impossible; the same laws that govern the weather apply to an organization.

However, there is an alternative to forecasting: visual management (also referred to as pull, lean or kanban).

Visual Management

Visual management was invented within production to solve the problems of wait time for materials due to errors in forecasts. When using outdated methods, if production cell A produces materials that are used by B a forecast is made of this use and A is allowed to produce according to the forecast. When there are many active production cells complex forecasts are created. In practice it’s impossible to make these kind of forecasts (see the book Chaos for an explanation as to why this is so). The forecast will lead to a lot of disruptions which forces management and workers to constantly run around, putting out fires. All this extra work is called waste.
On the other hand, visual management is based on how A can see by himself/herself how much material is used by B and produce more as needed (and not more). One example of this using boxes is when the material in one box is used, B sends the empty box to A to be refilled. Often a so-called kanban is used, so that when the material is down to a certain level B sends a kanban to A who produces the required materials.

Visual Management and Pulse

Pulse-activity-window

Visual management within strategy and development follows the same principles as production. A job’s current status is made visible using a whiteboard (a Pulse board) so that A and B can see the current situation. The status on the board tells A and B what they need to do in order for work to progress without wait times. The meeting where visual management is used is called a Pulse meeting. These Pulse meetings form a network and we refer to this as an agile network organization.

Workload Limits

A good system for visual management also needs to set limits for how much work can be done concurrently. The aim of setting such limits to keep the operation from overloading in order to keep the lead times short. Appropriate workload limits can be calculated using Little’s Law.