University of Minnesota
Software Engineering Center
/

You are here

Software Metrics - Starting at the Top

Date of Event: 
Thursday, January 9, 1997 - 1:00pm

The following are the questions posed during the discussion, and the results collected:

As a group, what do we use Metrics for?
What can they tell you?

  • Quality - What gets measured gets done.
  • Project Management.
  • Measure quantitatively the state of what's going on.
  • Tell how well we're doing.
  • Process Management.
  • Schedule.
  • Defects.
  • Justification.
  • Cost.
  • Cycle time.
  • Prediction.
  • Trends.
  • Decision making.
  • Process improvement.

Based on the list generated from "What do we use metrics for?", as teams, discuss and come to consensus on the top three uses.

  • Measuring Status (3)
  • Continuous (Process) Improvement (3)
  • Estimation (Prediction) (3)
  • Decision Making (Action) (Finding Trends) (3)
  • List of Test Cases

Can the same metrics be used for all purposes?

  • No - 3
  • Maybe - 1

As teams, discuss and come to consensus on what are the minimum things you should be measuring?

  • Cost (2)
  • Effort
  • Time (2), Schedule Milestones
  • Quality, Defects (2)
  • Size
  • Staff Size
  • Rate of code
  • Adherence to user requirements
The consensus pretty much tracked with my experience and the SEI recommended core measures: effort, duration, size, and quality.

Include a list of measures you have successfully used.

  • Requirements Stability (2)
  • Size Stability
  • Defect Density (2)
  • Defect Yield 2)
  • Defects vs. Time (2)
  • Milestones Hit/Missed (2)
  • Mean Time To Failure
  • Cost of Defect Correction (2)
  • Test Completion vs. Time
  • Cost
  • Size
  • Staff Size
  • Hours
  • Change Rate
  • McCabe Complexity

As a group, what are the advantages of not measuring, no plans, no statusing:

  • Can't be used against you, not accountable, can't be blamed.
  • Minimizes administrative overhead.
  • Less time communicating, less time wasted.
  • Do real work.
  • Done when you want.
  • No confusion about metrics.
  • Do what you want.
  • Sales do what "they" want.
  • Increased creativity.
  • Don't have to follow processes.
  • Not demoralized.
  • Not as much management "help"
  • More responsive to marketplace.
This is obviously a "trick" question. The above answers, which came fast and furious seem true at first, but when you really think about them, usually the opposite applies. For example, "Less time communicating, less time wasted" - up-front this might be true, but in the end a well planned and measured project will be more productive and have higher quality. People will know what to do, when to do it, and it is more likely to be what the customer wanted. What we are dealing with here is the "soft" or human side of measurement.

When it comes to measuring ourselves, our processes, our product, we have a built-in aversion. This is especially true when other people will be looking at the data. We tend to think of measuring only as a way to see how well we as individuals are doing, and most of us already do our best, so why do we have to be measured. We need to see and understand the "big" picture and how measurement and it's associated pieces (planning, estimating, statusing) help the project and the organization, and in the long run, us.

Still don't believe it's a soft issue? Look at the reasons the group came up with for measuring and planning.

As a group, what are the disadvantages of not measuring, no plans, no statusing:

  • Don't know how much trouble you are in.
  • Don't know when you are done.
  • Can't estimate the next project.
  • Can't tell if improving or getting worse.
  • Don't know if the software will work.
  • Decision making turns into a power struggle.
  • Can't justify any needs.
  • Always in a crisis mode, reacting vs. proactive.
  • Don't know if your resources are being used effectively.
  • Frustration.
  • Rework.
These are pretty much the opposite of the reasons given for not measuring. How can they both be true? Obviously they can't, and the first set is false.

So how do you minimize this natural tendency to avoid measuring and planning? We were running out of meeting time and my suggestion is to continue this topic in more detail next meeting. I would like to have about four people to volunteer to each present a 15 minute description on the metrics they are using in their organization today and then have a panel led group discussion on their presentations with an emphasis on what they did to minimize the soft issues involved with measuring. Please volunteer, the presentation need not be formal, just describe what you are currently doing.

Not to leave you hanging, I have included the bullets from my last two slides:

Dimensions of a Successful Measurement Program

  • Start small, define well, and automate.
  • Create a process improvement culture, involve all
  • stakeholders, realize it takes time.
  • Educate and train.
  • Build and maintain trust.
  • Experiment, be adaptable.
  • Use the data to understand and improve.

Conclusion

  • Don't measure individuals, measure process and product.
  • Don't ignore the data.
  • Never use only one metric.
  • Select metrics based on objectives.
  • Provide feedback.
  • Let the team select the metrics (buy-in).
  • Must be part of the culture, part of an overall process improvement program.