Fortnight Milestones

John Ousterhout
Usman Muzaffar

We have tried many approaches to managing schedules for the software projects on which we have worked and until recently we were never satisfied with the results: each project seemed to have a life of its own that bore little resemblance to the schedule, and there were always surprises and unexpected delays near the end of the schedule. However, during the development of the ElectricCommander product at Electric Cloud we tried a new approach to managing software schedules that we have come to call "fortnight milestones". The basic idea is to create schedule predictability by using short regularly-spaced milestones with well-defined goals. This technique worked quite well for us. It gave us good visibility and predictability for the project, and it helped us focus our efforts and maintain an appropriate level of urgency and intensity in development. Overall, it created a reliable heartbeat of progress, where both the engineers on the team and higher-level management could see continuous steady progress towards a predictable goal.

This article starts with some background on the goals and pitfalls of scheduling. Then it explains the fortnight milestone mechanism and discusses our experiences using it at Electric Cloud. Next, it shows how the microscopic predictability provided by fortnight milestones can be rolled up to provide visibility into the overall project schedule. Finally, the article compares fortnight milestones to Scrum sprints, which are similar in some respects.

Scheduling fundamentals

Ideally, a schedule serves two overall purposes in a software project. First and foremost, it should expose the truth: when will the project finish, and how much will it cost? This information allows management to make trade-offs between the features that will be delivered, the resources that will be required, and the delivery date. It also allows the organization to plan the other activities that surround a software project, such as marketing programs and sales rollout.

The second purpose of a schedule is to improve the execution efficiency of the team. An accurate schedule permits better resource planning: for example, resources can be moved from easier tasks to harder tasks so that they all finish at the same time. In addition, a schedule with appropriate deadlines encourages developers to work hard.

There are two problems that make it difficult to create an accurate schedule and manage it over the lifetime of a project. The first problem is that the schedule always underestimates the work. It simply isn't possible to predict at the beginning of a software project all of the detailed tasks that will have to be completed during the project. Key problems and issues often don't emerge until implementation is well underway; sometimes large components of the architecture turn out to be wrong and have to be redone. This is one of the reasons why the waterfall approach to software development has fallen out of favor; agile development recognizes this fundamental unpredictability and encourages a style of development that assumes things will change as the project evolves.

Although all schedules tend to be optimistic, shorter schedules are more reliable than longer ones. Most developers can predict fairly accurately what they can accomplish in a day or two. As the period of the schedule increases to a few weeks it becomes harder and harder to visualize all of the facets of the project; multi-month schedules are likely to contain significant errors. Thus a scheduling approach based on short-term schedules provides the best hope for predictability.

The second problem with managing schedules is that it is hard to gauge the status of a partially complete task: a status report of "80% done" means almost nothing. For many construction activities, such as building a brick wall or knitting a sweater, it is easy to measure progress: if the project looks about 80% done, then it probably is about 80% done. Unfortunately, this doesn't hold for software development. Surprises and delays almost always occur at the end of software development tasks, so a task that seems 80% done almost never is really 80% done; hence the numerous jokes about "the first 90% and the second 90%". The only status report that is truly believable is "100% done". If a software project has large tasks that have not completed yet, it is difficult to gauge the real status of the project.

Fortnight milestones: basic timing

In this section and the ones that follow we will describe the fortnight milestone mechanism. Each section is based around a few simple rules. The first rules describe the 2-week cycle of a milestone:

The traditional way to schedule a software project is to pick milestones based on the completion of major pieces of functionality, such as "catalog management complete" or "ready to accept credit cards" and then assign dates to those milestones. Fortnight milestones flip this on its head: milestones are chosen based purely on time (every other week) and then tasks are chosen to fit the milestone dates.

Why are milestones so short? Short milestones allow more precise scheduling of tasks within the milestone; most developers can estimate fairly accurately how much work they can get done over a 2-week period. Short milestones also expose problems quickly when tasks aren't completed on time. Finally, short milestones create a sense of urgency. In our experience the best way to get developers to work hard is to give them achievable goals, personal responsibility, and a near-term deadline; fortnight milestones provide all of these.

Why finish on Monday? If a developer is running behind on a task they will know it by the end of the second week. With a Monday deadline developers have the option of working over the weekend to catch up.

Why is the milestone duration fixed and uniform? There are two advantages to fixed-length milestones. First, they create a consistent rhythm to the project (see Figure 1). Everyone works hard during the second week of a milestone to finish before the deadline. During the first week of the next milestone developers can relax a bit, clean up loose ends from the previous milestone and start work on the next milestone. During the second week of the milestone the pressure builds again as the deadline nears. This "one week on, one week off" approach strikes a reasonable balance between working hard and having some time to relax and think.

Figure 1. Fortnight milestones settle into a rhythm where the first week of each milestone is relatively relaxed but the pace picks up during the second week as the deadline approaches.

The second advantage of fixed-length milestones is that they make it possible to compare results across different milestones to discover trends. We will discuss this more in a later section.

Why not make milestones even shorter? If 2-week milestones are good, wouldn't shorter milestones be even better, perhaps one week or even a single day? We considered shorter milestones but were concerned that they would result in nearly continuous deadline pressure, which would cause burnout over a long project. We also worried that shorter milestones would not provide enough time to make meaningful progress; on the other hand, two weeks is almost always enough time to complete something interesting. Finally, shorter milestones would eliminate the time periods where cleanup and careful design are most likely to happen, so this could impact the quality and architecture of the system. Overall, two weeks seemed to us the shortest workable period for a milestone.

Planning the tasks for a milestone

When choosing the goals for a particular milestone, the most important considerations are responsibility and predictability. Goals should not be shared between developers, since this can result in confusion over who is responsible for what. If a shared goal arises, see if you can divide it into multiple goals, one for each developer. Try also to avoid dependent goals where one developer cannot complete their goal unless another developer completes a different goal. If possible, move the dependent goal to a later milestone and have that developer work on something else without dependencies. Developers will work hardest and most effectively if they have specific goals and complete responsibility.

It is essential to pick goals where completion is unambiguous and achievable within the milestone; ideally completion is measured by a computer, for example by completing an automated build-and-test cycle. Remember that only 100% completion can be measured accurately. If a goal cannot be completely finished within the current milestone, subdivide it. For example, instead of "login mechanism 60% coded", pick a smaller goal that can be finished, such as "password class coded and tested". Completion means that there is no known additional work related to this task. In our environment developers are expected to write unit tests for all of their code, so completion means "coded, compiled, unit tests written with near-100% code coverage, and internal documentation written". Some of our developers use test-driven development, where they write tests first and then use the tests to drive the detailed coding. In any case, "finished except for the tests" is an oxymoron.

Ideally each developer should have 2-4 goals for each milestone, with each goal taking a couple of days; this is a size where developers can predict implementation time fairly accurately. It is almost always possible to subdivide larger goals into smaller ones that can be completed in a couple of days; the process of subdivision forces developers to think through the details and exposes additional tasks that had been overlooked. We also set a lower bound on goals: no matter how simple it seems, we force developers to budget at least a half day for each goal.

Where possible, we try to find a theme for each milestone that ties together all of the individual goals, such as "end-to-end transmission and presentation of one data record" or "user login". Themes create a sense that everyone is working together on something exciting, which encourages communication and increases motivation. However, themes need to be planned carefully to minimize dependencies; ideally each person can work individually on their own part of the theme, e.g. with test fixtures to simulate the other parts, and everything comes together at the end of the milestone.

Milestone transitions

It's not unusual for some of the goals to be incomplete at the end of a milestone. When this happens, do not extend to the milestone to finish the goals. Just record which goals were finished and which weren't, move on to the next milestone, and use the experience to improve the plans for future milestones.

Each milestone should be (re-)planned from scratch at the beginning of the milestone. We have a team meeting on the Tuesday after the end of each milestone to review the results of the previous milestone and plan the goals for the next milestone. When starting a new milestone it's tempting to take all of the unfinished work from the previous milestone and simply add it to what you had previously planned for the next milestone. We call this "schedule compression"; if you succumb to this temptation then milestones become increasingly overbooked to the point where they are totally unbelievable and no one takes them seriously. If you miss a milestone it means there was too much work allocated to that milestone, so you need to allocate less work to the next milestone, not more. If you miss a bunch of early milestones in a project, it probably means the overall project schedule is too optimistic. This may not be welcome news, but you might as well accept it and start planning accordingly.

The best approach is to reconsider each milestone from scratch; based on what you know now, what are the most important tasks to work on, and how much can be accomplished in the next two weeks? If you choose to finish the unfinished goals from the previous milestone then you will need to push out other things you might previously have planned for this milestone. Or, you might choose to delay completion of the previous goals until a later milestone, but even then you probably need to reduce the content of the new milestone so that you have a better chance of finishing everything.

Once you have decided on the goals for a milestone, don't change them during the milestone: replanning every 2 weeks is often enough. If you find yourself under external pressure to change goals in the middle of a milestone, it suggests an overall organizational issue that you should address.

Recording and tuning

The time estimate for each goal should come from the developer who will carry out the task. Bear in mind that developers tend to underestimate these times, and peer pressure at all levels tends to produce over-optimistic schedules. We have heard rumors of developers who consistently "sand-bag" with overestimates, but neither of us has actually seen such a developer. Thus, unless a developer has a history of overestimation you should never suggest that they reduce an estimate, no matter how high it seems to you and no matter how unpleasant the consequences of that estimate for the overall project; this will just set the developer up for failure. On the other hand, you should definitely encourage developers to increase an estimate that seems low. If it seems too low, it probably is; we sometimes use a rule where we pick the highest estimate suggested by anyone around the table and even so the estimates are rarely too high. If you want a collection of tasks to finish on time then some of them must finish ahead of schedule, since there will certainly be others that take longer than expected. The most important thing is accuracy: the goal is to get into a rhythm where the team consistently meets most or all of the goals for each milestone.

In our projects we use a page in our Wiki to manage our milestones, but almost anything will work; there's no need for fancy infrastructure. We record the goals at the beginning of each milestone, one line per goal with a very short description, developer name, and time estimate. At the end of each milestone we note whether the goal was met. This encourages a sense of responsibility, since everyone can see who is responsible for what and their success or failure will be displayed publicly. Furthermore, we use the accumulated data to improve our planning in the future: it's easy to skim over recent milestones to see if we are overcommitting and identify developers who are consistently underestimating. The milestone trends also help management to get a feel for overall project progress; more on this a bit later.

Our experiences at Electric Cloud

The mechanism for fortnight milestones evolved during the initial development of the ElectricCommander product at Electric Cloud. This was a relatively small project with 4-6 developers. The 1.0 release involved about 20 milestones and subsequent releases have involved 6-10 milestones each. There were typically about 10 goals for each milestone.

We were quite happy with the results produced by this approach. We typically completed 70-80% of the individual goals for each milestone. We occasionally completed 100% of the goals for a milestone, but usually there were one or two goals that weren't completely finished. This felt about right to us; we like stretching just a little bit with our goals. Occasionally we had major milestone misses where we completed less than 50% of our goals; when this happened, it was usually because we didn't plan the milestone carefully enough. It's important for developers to take a few minutes to think about their goals for the next milestone before coming to the milestone planning meeting; it's tempting to make off-the-cuff guesses in the meeting, but these tend to be inaccurate.

Overall, fortnight milestones produced predictable, steady results for the development team. This was good both for the morale of the team and for management confidence. As with almost all projects, our initial hopes for what we could accomplish in each release were overoptimistic but this became clear after a few milestones so we could adjust our expectations and focus on the most important features. As a result, there have not been schedule surprises at the ends of release cycles.

Fortnight milestones also served as a good motivational tool. The frequent deadlines encouraged us to work hard, but at a sustainable pace that could continue month after month without burnout.

Finally, fortnight milestones helped the project to maintain focus. In a traditional schedule with long milestones, time seems plentiful in the early days so it's easy to waste time on nonessential features or overengineering of the early projects. With fortnight milestones we asked ourselves every two weeks what are the absolutely most important things to work on. This encouraged us to defer nonessential tasks and focus on the big issues; every task of more than a few hours duration gets reviewed by the entire team.

Estimating overall project completion

With fortnight milestones you should be able to achieve predictable progress at the microscopic level of 2-week milestones, but this doesn't guarantee that your overall project will finish on time with the desired feature set. Even if you consistently meet your milestone goals, there could be other factors that delay the overall project: your initial planning may have omitted important tasks or underestimated implementation time; you may discover problems during implementation that require significant redesign; or external factors may require you to change the feature set for the release. Fortnight milestones cannot prevent these problems from occurring. However, fortnight milestones do provide a predictable base that helps to expose problems such as these and quantify their impact, so you can plan accordingly.

The key to overall project predictability is to create an accurate record of past progress, which you can use to predict future progress. Fortnight milestones provide the fine-grain historical data needed for this.

At the beginning of the project, do the best you can to divide the project into tasks no larger than a couple of days and make an initial time estimate for each task in engineer days. The accuracy of this information will depend on how much time you have to spend on the initial design, the complexity of the project, and how similar the tasks are to things you have done before. Then track the evolution of the estimates as milestones complete. Figure 2 shows how a spreadsheet can be used to track a project on a milestone-by-milestone basis.

Figure 2. An example of a spreadsheet that records the evolution of a project's schedule. Each row in the spreadsheet represents one task, which will be eventually be a goal in one or more milestones, and each column records the estimated work remaining for that task on a particular date, in engineer-days. A new column is added at the end of each milestone, and new rows are added as new tasks are discovered. Once a column is added it is never modified. Green cells indicate task completion, and yellow cells indicate the discovery of additional work. The bottom row ("Total Work Remaining") sums all of the estimates in the column above it and provides the key indicator of overall progress. By observing the rate at which total work remaining is dropping you can estimate when the project will complete. For a complete version of this spreadsheet, click here.

The key overall metric is the total estimated work remaining; you should be able to compute a new value for this after each milestone. Once you have completed a few milestones you can use the trend in total work remaining to estimate the project completion date. For example, suppose the work remaining is dropping by about 15 engineer days per milestone; if the total work left is 60 days then it will take about four more milestones, or two months, to complete the desired feature set. Note that it doesn't matter how many developers you have on the project; all that matters is the rate at which the apparent work is dropping.

Most likely your initial schedule was both optimistic and incomplete. The milestone data will quickly expose this (work remaining won't drop as fast as you would expect from the number of engineers on the project) and the estimation approach from the previous paragraph will take this into account. In the example above, if there are 3 engineers on the project it would be tempting to conclude that 60 days of work remaining means 20 days per engineer, or two milestones. However, the historical data shows that the schedule was optimistic by a factor of two (three engineers are only completing 15 days of work every two weeks), so it will probably take four milestones, not two, to complete the project.

If the total work remaining is not dropping at a steady rate, it indicates one of two problems:

• The overall schedule is out of control. Perhaps you didn't anticipate all of the tasks, so that new tasks are constantly appearing. Or, perhaps your initial time estimates were incorrect. Or, perhaps the overall project scope is changing (e.g., new features are being added). The milestone records will help you to figure out which of these is the problem, so you can address it.
• You are not consistently meeting milestone goals. If this is the case you must set more conservative goals for future milestones. Your milestone records will indicate whether there are issues across the board or only with certain developers or sub-projects.

The key aspect of this approach to project-level scheduling is to record historical data and use actual results from the past to predict future performance. Traditional scheduling techniques tend not to keep historical data or they keep it on such a coarse grain that it is hard to extrapolate. As a result, predictions for the future tend to be based on hope or intuition rather than measured results. Fortnight milestones provide fine-grain data that gets the truth out early; this often results in bad news early in the schedule, but it reduces the likelihood of bad news at the end of the schedule.

Life before fortnight milestones

Before implementing fortnight milestones we used a more traditional approach to scheduling with only a few large milestones per release. In some cases, the first milestone was the alpha release. Schedules tended to be focused around individual developers rather than project-wide checkpoints.

In comparison to fortnight milestones our previous approach had several drawbacks. The schedule tended to get fixed at the beginning of the project and was not revised, so schedule compression occurred: when early tasks took longer than expected, we somehow managed to convince ourselves that we could make up the time during later tasks. Of course, this rarely happened. We tended not to face up to problems until shortly before the first alpha release. There was little feedback on how we executed against the schedule, so developers didn't learn how to make better time estimates; often the schedule was made and then ignored. Developers felt little pressure until the first hard deadline (typically the alpha release) and then suffered from continuous pressure for the rest of the release as problem after problem appeared. Management had little visibility into the progress of the project, and little confidence.

When not to use fortnight milestones

Fortnight milestones will not necessarily work for every project or for every phase of a project. In particular, we don't recommend using them in situations where the goals are changing daily. For example, the alpha and beta periods for a project are often highly reactive, where new bugs arrive daily and it's important to fix them quickly so that customers can continue their testing. In this situation no form of schedule is likely to work; the best approach is to drive development with a prioritized issue list rather than a schedule; consider having short daily standup meetings to reprioritize efforts based on what has happened in the last 24 hours.

Fortnight scheduling can work well for projects whose focus is to fix bugs rather than add new features, as long as the bug list is relatively stable. If you find yourself having to change priorities in the middle of a milestone, then your situation may be too volatile to use fortnight milestones (or any scheduling).

Fortnight milestones vs. Scrum

Fortnight milestones are similar in many ways to sprints as used in the Scrum method for project management. However, fortnight milestones fix the interval for each milestone at exactly 2 weeks, whereas Scrum sprints are chosen based on features. Scrum sprints can vary in duration and are typically longer than 2 weeks. Making milestones short and uniform in length creates a consistent rhythm and makes it easier to compare results from different milestones to identify trends.

Conclusion

There is no simple and guaranteed solution to the problem of managing complex software projects. It will always be hard to plan projects accurately, and there will always be surprises after implementation starts. However, we found that fortnight milestones minimize the surprises and create a predictable heartbeat of engineering that leads to confidence for both developers and management.

Overall, fortnight milestones accomplish two things. First, they expose the truth about how a project is progressing. This may not always be pretty, but it allows management to make adjustments and helps to prevent disasters at the end of the development cycle. Second, fortnight milestones help the team to execute more efficiently by providing continuous feedback, by generating an appropriate sense of urgency, and by encouraging focus.

If you find a fortnight milestones interesting enough to try in your own projects, we would be interested to hear about your experiences. Send an e-mail to John Ousterhout to let us know whether they worked for you and what you did and didn't like about them.