173
Chapter 23. Adaptive Software Development
Collaboration is difficult, especially when it involves other people.
—Ken Orr (Cutter Consortium Summit 2001)
In 1992, I started working on a short-interval, iterative, RAD process that evolved into Adaptive
Software Development. The original process, developed in conjunction with colleague Sam Bayer,
was used to assist in marketing a mainframe RAD tool. Sam and I worked with prospects on pilot
projects—one-month projects with one-week iterations—in companies from Wall Street
brokerage houses to airlines to telecommunications firms. Over the next several years, Sam and I
(together and separately) successfully delivered more than 100 projects using these practices, and
in June 1994, we published an article on our experiences (Bayer and Highsmith 1994). During the
early to mid-1990s, I also worked with software companies that were using similar techniques on
very large projects, while Sam continued to evolve the practices in his work.
In the mid-1990s, my interest in complex adaptive systems began to add a conceptual background
to the team aspects of the practices and was the catalyst for the name change from RADical
Software Development to Adaptive Software Development (Highsmith 1997). ASD has been used
by companies from New Zealand to Canada for a wide range of project and product types.
Interestingly, I became aware of XP just a month prior to the publication of Adaptive Software
Development (Highsmith 2000), when Kent and I exchanged emails. Extreme Programming
Explained (Beck 2000) and Adaptive Software Development were published within a couple of
months of each other.
Complexity theory helps us understand unpredictability and that our inability to predict doesn't
imply an inability to make progress. ASD works with change rather than fighting against it. In
order to thrive in turbulent environments, we must have practices that embrace and respond to
change—practices that are adaptable. Even more important, we need people, teams, and
organizations that are adaptable and Agile. Agile practices alone are not nearly enough; they
depend on competent individuals who are nimble and thoughtful. We have become so enamored
of precise planning, we forget that products evolve from a little planning and a lot of learning as
we proceed.
For all the many books on requirements engineering, the best way to determine how a product
should evolve is to use it. Iteration—building, trying, succeeding, failing, rebuilding—governs
successful product development, particularly in extremely competitive environments. "Good
enough" requirements need to be followed by quick delivery and use, and then followed with
evolutionary changes to the requirements and the product based on that use. Although a number of
modern software development life cycles have adopted an iterative approach, they still miss the
mark in dealing with the messiness of complex environments. Despite the fact that development is
iterative, many people's fundamental assumptions are still deterministic—they think of short
waterfall life cycles strung together.
The practices of ASD are driven by a belief in continuous adaptation—a different philosophy and
a different life cycle—geared to accepting continuous change as the norm. In ASD, the static plandesign-build
life cycle is replaced by a dynamic Speculate-Collaborate-Learn life cycle (see
Figure 23.1). It is a life cycle dedicated to continuous learning and oriented to change,
reevaluation, peering into an uncertain future, and intense collaboration among developers,
management, and customers.
Figure 23.1. The Speculate-Collaborate-Learn Life Cycle 174
A Change-Oriented Life Cycle[1]
[1] Some material in this chapter has been adapted from the article "Retiring Lifecycle Dinosaurs"
(Highsmith 2000a).
A waterfall development life cycle, based on an assumption of a relatively stable business
environment, becomes overwhelmed by high change. Planning is one of the most difficult
concepts for engineers and managers to reexamine. For those raised on the science of
reductionism (reducing everything to its component parts) and the near-religious belief that careful
planning followed by rigorous engineering execution produces the desired results (we are in
control), the idea that there is no way to "do it right the first time" remains foreign. The word
"plan," when used in most organizations, indicates a reasonably high degree of certainty about the
desired result. The implicit and explicit goal of "conformance to plan" restricts a manager's ability
to steer the project in innovative directions.
"Speculate" gives us room to explore, to make clear the realization that we are unsure, to deviate
from plans without fear. It doesn't mean that planning is obsolete, just that planning is
acknowledgeably tenuous. It means we have to keep delivery iterations short and encourage
iteration. A team that "speculates" doesn't abandon planning, it acknowledges the reality of
uncertainty. Speculation recognizes the uncertain nature of complex problems and encourages
exploration and experimentation. We can finally admit that we don't know everything.
The second conceptual component of ASD is collaboration. Complex applications are not built,
they evolve. Complex applications require that a large volume of information be collected,
analyzed, and applied to the problem—a much larger volume than any individual can handle by
him- or herself. Although there is always room for improvement, most software developers are
reasonably proficient in analysis, programming, testing, and similar skills. But turbulent
environments are defined in part by high rates of information flow and diverse knowledge
requirements. Building an eCommerce site requires greater diversity of both technology and
business knowledge than the typical project of five to ten years ago. In this high-information-flow
environment, in which one person or small group can't possibly "know it all," collaboration skills
(the ability to work jointly to produce results, share knowledge, or make decisions) are paramount.
Once we admit to ourselves that we are fallible, then learning practices—the "Learn" part of the
life cycle—become vital for success. We have to test our knowledge constantly, using practices 175
like project retrospectives and customer focus groups. Furthermore, reviews should be done after
each iteration rather than waiting until the end of the project.
An ASD life cycle has six basic characteristics:
1. Mission focused
2. Feature based
3. Iterative
4. Time-boxed
5. Risk driven
6. Change tolerant
For many projects, the requirements may be fuzzy in the beginning, but the overall mission that
guides the team is well articulated. (Jens Coldeway's insurance project discussed in Chapter 11 is
a good example of this.) Mission statements act as guides that encourage exploration in the
beginning but narrow in focus over the course of a project. A mission provides boundaries rather
than a fixed destination. Without a good mission and a constant mission refinement practice,
iterative life cycles become oscillating life cycles—swinging back and forth with no progress.
Mission statements (and the discussions leading to those statements) provide direction and criteria
for making critical project tradeoff decisions.
The ASD life cycle focuses on results, not tasks, and the results are identified as application
features. Features are the customer functionality that is to be developed during an iteration. While
documents (for example, a data model) may be defined as deliverables, they are always secondary
to a software feature that provides direct results to a customer. (A customer-oriented document
such as a user manual is also a feature.) Features may evolve over several iterations as customers
provide feedback.
The practice of time-boxing, or setting fixed delivery times for iterations and projects, has been
abused by many who use time deadlines incorrectly. Time deadlines used to bludgeon staff into
long hours or cutting corners on quality are a form of tyranny; they undermine a collaborative
environment. It took several years of managing ASD projects before I realized that time-boxing
was minimally about time—it was really about focusing and forcing hard tradeoff decisions. In an
uncertain environment in which change rates are high, there needs to be a periodic forcing
function to get work finished.
As in Barry Boehm's spiral development model, the plans for adaptive iterations are driven by
analyzing the critical risks. ASD is also change tolerant, not viewing change as a "problem" but
seeing the ability to incorporate change as a competitive advantage.
The Basic ASD Life Cycle
Figure 23.2 shows an expansion of the ASD life cycle phases into specific practices.
Figure 23.2. The Adaptive Life Cycle Phases 176
Speculate: Initiation and Planning
There are five general steps in "speculating," although the word "steps" is somewhat of a
misnomer, as each step may be revised several times during the initiation and planning phase.
First, project initiation involves setting the project's mission and objectives, understanding
constraints, establishing the project organization, identifying and outlining requirements, making
initial size and scope estimates, and identifying key project risks. Because speed is usually a major
consideration in using ASD, much of the project initiation data should be gathered in a
preliminary JAD session. Initiation can be completed in a concentrated two- to five-day effort for
a small- to medium-sized project or take two or three weeks for larger projects. During the JAD
sessions, requirements are gathered in enough detail to identify features and establish a skeletal
object, data, or other architectural model.
Next, the time-box for the entire project is established based on the scope, feature set requirements,
estimates, and resource availability that result from project initiation work. Speculating