Thad Scheer
SPHERE OF INFLUENCE, INC. – software studios and services
Capacity estimation for software development
There is no perfect method for funding or estimating new product development, and that includes software projects. That said, Capacity Estimation is pretty good…
Background:
Labor = cost when developing software, so most approaches to estimation lean heavily on Level of Effort (LOE) analysis, which uses work breakdown as the backbone for estimating cost.
The problem is that inputs such as number of tasks, productivity, and complexity have a high standard deviation of error depending on the climate of the estimation process and the person doing the estimating. Consequently, cost estimates for new product development, and software development in particular, are usually off by a mile.
People disagree about how to fix this. One camp believes analytical rigor is the solution and suggests that people should put more energy into better measurements and baselines. This camp argues that if we can measure it, we can predict it.
Not everyone agrees, many say that dog will never hunt. The primary evidence is the decades people have spent working with detailed analytical costing models with poor industry-wide success.
Most software development is one-off, and the challenges facing a team on Project-A may have little to do with what they face on Project-B. Moreover, intense measurement is costly and raw data is useless by itself. Metrics need to be reduced to be useful and the method of analytical reduction (the “model”) will usually have enough knobs on it that you can dial in any result you want, regardless of the data. The whole estimation process becomes what our SOI-GOV President, Tab Burton, has called a “self licking ice cream cone”.
The most forward leaning people I know are giving up on analytical models for cost and resource allocation on new product development. The trend is toward capacity funding models.
What is a capacity model?
The idea is to fund development around capacity instead of tasks. Capacity can be programmed different ways, one approach is to use a percentage of the revenue potential; whereby products with larger revenue potential get more capacity. Capacity is generally rounded to "shirt sizes" for purposes of funding, and the details are captured in the "capacity profile". Capacity profiles are pre-set for the organization and vary depending on the type of work. There might be a profile for Application Product Development and another profile for Scientific Computing. The profiles specify what a "normal team" looks like for each shirt size in terms of creative talent and other resources.
This simplifies the budgeting process significantly. Managers fund capacity, not tasks, stories, milestones, function points, or anything else that offers precision without accuracy.
Lots of people are experimenting with ways to program capacity for single products as well as portfolios. For example, Jochen Krebs at AOL uses relative revenue potential as the shoreline for setting capacities across their extensive portfolio of projects.
Where’s the accountability?
By using capacity instead of funding tasks and milestones, we are funding...well...capacity; so where is the accountability? What ensures that capital deployments are productive?
As with any other product development approach, reviews and checkpoints are essential. There are many ways to do this. Classic risk management approaches use discrete gate-based checkpoints whereas modern Agile approaches create an environment of continuous accountability.
With all new product development, if you don't push a product team there is no guarantee they will perform. Also, if you don't apply demanding standards throughout development you might get garbage. This is true regardless of whether capacity funding is used or not.
If return < investment then the project needs to be canceled. In actuality, a threshold should trigger long before that. For example, an executive funds a "size-small" capacity effort for a product development profile to build a customer-facing web application...and the first release is set for 6 months from start date. If either the date is missed or the revenue from the first release does not meet the minimum threshold, cancel the effort and disband the team.
Incremental funding: When each round of capacity funding is perceived by the delivery team to be "at risk", this is an effective managerial weapon, use it! If a delivery team calls your bluff...don't blink - cancel the project.
Applicability to government projects: A topic for another post.