A brief history of agile methods

2 minute read by Erin 'Folletto' Casali

There is a common misconception that agile methods are just for smaller projects. While it’s obvious the falsity of this statement to anyone that actually did it properly, it’s also obvious when we check the origins of agile.

  • 1930s — Walter Shewhart proposes a series of short “plan-do-study-act” (PDSA) cycles.
  • 1950s — The X-15 hypersonic jet applied incremental and iterative development.
  • 1958 — Project Mercury (NASA) software development, ran with half-day iterations.

“All of us, as far as I can remember, thought waterfalling of a huge project was rather stupid, or at least ignorant of the realities.”
— Weinberg G. M. (Project Mercury)

“Every one of those deliveries was on time and under budget”
— Mills H.

“Software development should be done incrementally, in stages with continuous user participation and replanning and with design-to-cost programming within each stage.”
— Mills H. (1976)

  • 1977-1980 — Space Shuttle (NASA) avionic software. 17 iterations over 31 months (8 weeks average).
  • 1980s — Artificial intelligence researchers used Lisp machines and evolutionary prototyping.
  • 1987 — Command and Control Processing and Display System Replacement, developed by TRW in 6 time-boxed iterations.
  • 1980s — The DoD was experiencing a project failure rate of 75% in a sample of waterfall project of about $37 billion overall, where only 2% of them were used without extensive modification. At the end of 1987 the DoD changed its policies to allow iterative development.
  • 1994 — The DoD was still victim of the waterfall mindset, developing too much using waterfall and so Paul Kaminsky issued a report stating: “DoD must manage programs using iterative development”.

But of course, by the 90s the agile concepts were spreading more and as such more and more project were started using this approach, until in 2001 the agile manifesto was written.

Here I did just a summary, but the data I presented comes from a 2003 article by Larman and Basili, published by IEEE: “Iterative and Incremental Development: A Brief History” (PDF). It’s well written and with loads of references and details. Thanks to Dave Gray for pointing me to this great source.

What’s incredibly interesting is that the author of what’s considered the first formalization of waterfall, Winston Royce, agrees instead with incremental and iterative development. Even in its very first article in 1970, “Managing the Development of Large Software Systems” (PDF), he states:

STEP 3: DO IT TWICE
If the computer program in question is being developed for the first time, arrange matters so that the version finally delivered to the customer for operational deployment is actually the second version insofar as critical design/operations areas are concerned.

Waterfall… has never been really waterfall.

Why then? Well, if you think about it, it’s all about human perception and learning. The waterfall article dedicated entire pages to this “STEP 3: DO IT TWICE” but unfortunately it wasn’t part of the core message in the “waterfall” idea, and as such dropped out in the popular understanding of it.

It’s a matter of perception, learning and communication.

That’s also why I started talking about the Dot Loop: to have a simple and effective way to communicate the approach I value most and show how it applies universally.

Erin 'Folletto' Casali 2012·03·19