Good, Fast (Now), and Cheap: Pick All Three
New! Listen to the MP3 Podcast version of this article
Company owners, sales professionals, and software development professionals all want to deliver high-quality products and services to their customers. They want to do this more quickly than their competition, reducing their time-to-market. They also want to generate a return-on-investment (ROI) as soon as possible. To top it off, they want to do all of this at a lower cost of development to maximize profitability.
To successfully deliver products and services, software development professionals have designed what are called “software development methodologies”. Essentially, a methodology is a collection of practices, policies, and conventions used to plan, develop, and deliver a software system over the lifecycle of a project.
Article Goal: Use and Understand Both Waterfall and Agile Methods in the Next 15 Minutes
Two of the most widely known and used types of methodologies are broadly classified as “waterfall” and “agile”. The goal of this article is not to provide an exhaustive theoretical treatise on methodologies, but instead to simulate both waterfall and agile by getting you the reader to apply each of them to process of reading the remainder of this article. By doing this, you will experience a microcosm of what it feels like to work with each type of methodology.
First, let’s get some basic definitions. These from Wikipedia will do just fine:
Waterfall software development
The waterfall model is a sequential software development process, in which progress is seen as flowing steadily downwards (like a waterfall) through the phases of Conception, Initiation, Analysis, Design (validation), Construction, Testing and maintenance.
Agile software development
Agile software development refers to a group of software development methodologies based on iterative development, where requirements and solutions evolve through collaboration between self-organizing cross-functional teams. The term was coined in the year 2001 when the Agile Manifesto was formulated.
Let’s use both of them now and compare and contrast them.
First Let’s Try a Waterfall-like Method to Read About Waterfall
In his book Software Estimation, software development legend Steve McConnell states that the first rule of estimation is to “Distinguish between estimates, targets, and commitments”. What tends to happen on many sequential Waterfall projects is that there is no distinction whatsoever.
Instead, what usually happens is that someone, usually a business owner will say, "We need to have this project done in three months, 90 days. We won’t have time for quality assurance on this project until the end, so don’t make many mistakes. Do you think you can make that date?”
The development staff, or, more likely, the appointed technical lead of the development staff will say, “We should be able to make that date.”
What just happened is that the business owner just heard a commitment from the developer. The developer may actually view that date as more of an estimate or a target. But, the damage is done. The owner now believes the product or service will be ready-to-ship in 90 days.
Now, regardless of the complexity of the work, the development staff will begin laboring to meet an arbitrary date chosen only by the owner and the technical lead. What’s worse, when quality assurance is “put off” to the end on projects like this, inevitably the project will run far longer than the desired and committed date. With no formal feedback loop in process until the end, the owner will likely “check in” periodically with the development team and ask the team lead, “How are we looking for meeting the three month date?” The team lead will tell him or her, “We’re doing pretty good”. What else can they say? They have not had any QA performed against their work, so it’s actually impossible for the development team to determine the correctness or completeness of their work!
You Have Exactly 90 Seconds, Now Go
So, to give you a chance to experience how this feels, I’ll play the role of the business owner who operates a web site that offers podcasts and recorded audio about software methodologies. I come to you, my team lead, and say:
“I need you to record yourself reading the following Wikipedia entry about Waterfall software development aloud, completely, and I need you to do it in one-and-a-half minutes, 90 seconds. I want you to read it carefully because we don’t have time for correcting mistakes. We need it ready for release on our downloads section of the web site. Do you think you can do that?”
Get out your stop-watch and time yourself!
Ready, Set, Read Aloud!
The waterfall model is a sequential software development process, in which progress is seen as flowing steadily downwards (like a waterfall) through the phases of Conception, Initiation, Analysis, Design (validation), Construction, Testing and maintenance.
It should be readily apparent that the waterfall development model has its origins in the manufacturing and construction industries; highly structured physical environments in which after-the-fact changes are prohibitively costly, if not impossible. Since no formal software development methodologies existed at the time, this hardware-oriented model was simply adapted for software development.
Owner: “How we doing?”
Team Lead: “Pretty good…”
Owner: “That’s great!”
The first formal description of the waterfall model is often cited to be an article published in 1970 by Winston W. Royce (1929–1995), although Royce did not use the term "waterfall" in this article. Ironically, Royce was presenting this model as an example of a flawed, non-working model (Royce 1970). This is in fact the way the term has generally been used in writing about software development—as a way to criticize a commonly used software practice.
In Royce's original Waterfall model, the following phases are followed in order:
1. Requirements specification
2. Design
3. Construction (AKA implementation or coding)
Owner: “Are we done yet?”
Team Lead: “How much of the last two paragraphs do we need finished before we release? We can’t make 90 seconds…”
Owner: “We need it all. I’m very disappointed, but this means we have to let the date slip. How much longer is this going to take?”
Team Lead: “Shouldn’t be too much longer…”
4. Integration
5. Testing and debugging (AKA Validation)
6. Installation
7. Maintenance
To follow the waterfall model, one proceeds from one phase to the next in a purely sequential manner. For example, one first completes requirements specification, which are set in stone. When the requirements are fully completed, one proceeds to design. The software in question is designed and a blueprint is drawn for implementers (coders) to follow — this design should be a plan for implementing the requirements given. When the design is fully completed, an implementation of that design is made by coders. Towards the later stages of this implementation phase, separate software components produced are combined to introduce new functionality and reduced risk through the removal of errors.
Owner: “How we doing now? Are we ready for QA?”
Team Lead: “Just a little while longer…”
Thus the waterfall model maintains that one should move to a phase only when its preceding phase is completed and perfected. However, there are various modified waterfall models (including Royce's final model) that may include slight or major variations upon this process.
All Washed Up
If you read aloud closer to how I read aloud, then it probably took you between 2 and 3 minutes (120 and 180 seconds) to read this passage. That’s a lot longer than 1.5 minutes or 90 seconds!
No wonder the owner is disappointed. He asked you if you could complete the assignment in 90 seconds. You told him you should be able to do so, and then you failed to deliver.
There’s got to be a better way to guess how long it’s going to take to do something we’ve never done before, right?
Now Let’s Try an Iterative and Incremental (Agile) Method to Read About Agile!
This time, we’re going to do something different to plan our release. As a business owner, I’ve learned my lesson from above and I realize I need more control over my release date and my teams. Because of this, I’m going to work with you to iteratively plan the release, based upon an actual, observed rate of progress. I’m going to call this rate of progress a “Sentence Velocity”. In later articles, we’ll read about software author Mike Cohn’s writings about user stories, story points, and velocity and how those are crucial measures of progress in an Agile world. But, since we’re starting small, we’re just going to worry about one sentence at a time.
Let’s Plan and Measure as We Go This Time
I’ve numbered each sentence or sentence equivalent sequentially. This time, I tell you:
“I want you to read the following passages from http://www.agilemanifesto.org aloud in 20-second iterations. Read the numbers out loud too. I will keep a tally of how many sentences you've gotten through after each iteration. I’ll divide this tally by the number of iterations, and this will tell me your Sentence Velocity per iteration. This way I can forecast how many iterations total it will take you to finish the whole page.”
On Your Mark, Get Set, Go!
Manifesto for Agile Software Development (1)
We are uncovering better ways of developing software by doing it and helping others do it. (2)
Through this work we have come to value: (3)
- Individuals and interactions over processes and tools (4)
- Working software over comprehensive documentation (5)
Owner looks at backlog, sees the team has read 5 sentences and calculates that it should take 6 iterations to complete, or 120 seconds.
- Customer collaboration over contract negotiation (6)
- Responding to change over following a plan (7)
That is, while there is value in the items on the right, we value the items on the left more. (8)
- Kent Beck
- Mike Beedle
- Arie van Bennekum
- Alistair Cockburn
- Ward Cunningham (9)
Owner again inspects the backlog, this time seeing 9 sentences finished, and calculated the pace is now 7 iterations, or 140 seconds. He plans his marketing, sales, training, and operations teams expectations accordingly.
- Martin Fowler
- James Grenning
- Jim Highsmith
- Andrew Hunt (10)
- Ron Jeffries
- Jon Kern
- Brian Marick
- Robert C. Martin (11)
- Steve Mellor
- Ken Schwaber
- Jeff Sutherland
- Dave Thomas (12)
Principles behind the Agile Manifesto (13)
We follow these principles: (14)
Owner observes the backlog and sees the velocity is still 7. Takes no action.
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. (15)
Welcome changing requirements, even late in development. (16) Agile processes harness change for the customer's competitive advantage. (17)
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. (18)
Business people and developers must work together daily throughout the project. (19)
Owner notes velocity is still 7. No action necessary.
Build projects around motivated individuals. (20) Give them the environment and support they need, and trust them to get the job done. (21)
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. (22)
Working software is the primary measure of progress. (23)
Owner notes velocity is still 7. Reminds other teams to prepare for the release.
Agile processes promote sustainable development. (24) The sponsors, developers, and users should be able to maintain a constant pace indefinitely. (25)
Continuous attention to technical excellence and good design enhances agility. (26)
Simplicity--the art of maximizing the amount of work not done--is essential. (27)
Owner notes velocity is still 7. The backlog has only two items now. Owner reminds everyone of the upcoming successful release.
The best architectures, requirements, and designs emerge from self-organizing teams. (28)
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. (29)
The team delivers the project after 7 iterations, as forecasted as early as the second iteration!
Reflection
When I did this in 20-second-iterations, I recorded the following sentence tallies after each iteration:
- 5
- 9
- 14
- 19
- 23
- 27
- 29
Suppose I wanted to calculate after the first iteration how many more iterations it would take me to complete. Well, I would simply divide 29 by my current velocity of 5 and get 5 with a remainder of 4. Well, there’s no such thing as a partial interval, so I'd know I need 6 iterations.
What about after 2? My velocity would now be 9 / 2, or 4.5 “Sentence Points” per iteration. 29 divided by 4.5 is 6.444, which rounds to 7 iterations.
My Sentence Velocity and Duration Forecast
Here’s how my sentence velocity and estimated total number of iterations after each iteration:
| Iteration | Sentences Completed | Velocity | Est. # Iterations |
| 1 | 5 | 5 | 6 |
| 2 | 9 | 4.5 | 7 |
| 3 | 14 | 4.67 | 7 |
| 4 | 19 | 4.75 | 7 |
| 5 | 23 | 4.6 | 7 |
| 6 | 27 | 4.5 | 7 |
| 7 | 29 | 4.14 | 7 |
As you can see, by the second iteration, I had a velocity that reliably predicted my actual number of iterations. In reality, requirements change, and velocity goes up and down, but it averages out over time. Mike Cohn recommends taking a moving average to calculate a team's velocity.
In the next article we’ll more about how Agile teams often create a “burndown chart” to visually indicate how much work remains after each iteration.
Conclusion
We’ve reviewed the two most prevalent software methodology styles, waterfall, and agile, and we’ve applied them to the reading process of this very article. In doing so, we first saw that it’s unwise to make a commitment to deliver a complete product on an arbitrary date. Next, we demonstrated a better way of planning by measuring our rate of progress, our velocity, and using that to forecast our project’s duration.
Until next time, stay agile not fragile.
References
- The Waterfall model: http://en.wikipedia.org/wiki/Waterfall_model
- Agile development: http://en.wikipedia.org/wiki/Agile_software_development
- The Agile Poster from VersionOne.com: http://pm.versionone.com/AgilePoster.html
