University of Minnesota
Software Engineering Center
/

You are here

Return on Investment (ROI) for SPI

Date of Event: 
Thursday, December 5, 1996 - 12:45pm

HO, HO, HO!

The following is Gail Bertossi's exquiste write-up on our December meeting:

A special thanks to Gail and to Burke.

December 5, 1997 TwinSPIN meeting


Attendees: 14
Topic: Return on Investment of Software Process Improvement
Speaker: Burke Maxey
(Presentation Summary is at end of this message.)
Sponsor: Benchmark Laboratories - Chuck LeCount December 1996
Next Meeting: January 9 (2nd Thursday of the month)
Next Topic: Metrics
Speaker: Jesse Freese of Fissure, Inc.
Sponsor: Honeywell Military Avionics - Dennis Robison January 1997
Suggested topic for March Meeting: dry run of Tim Olson's Metrics Framework presentation for the SEPG Conference

Visit our TwinSPIN web page: http://www.millcomm.com/~glewwe/TwinSPIN/welcome.html

Join our TwinSPIN Chat Line, contact: Gordon.Dosher@Network.com or 612-424-1770


ANNOUNCEMENTS

SEI's November, 1996 "Process Maturity Profile of the Software Community" (27 pgs) is available. (The reports produced twice a year are available for the past 3 years.)

URL: http://www.sei.cmu.edu/technology/measurement/profile.html

As an alternative, you can download the report from SEI's ftp site: ftp://ftp.sei.cmu.edu/pub/Empirical_Methods/Maturity_Profile/1996nov.pdf
(Note - the report is a pdf document and requires Adobe Acrobat.)

Magdy Hanna distributed a Call for Presentations for the local PSQT '97 conference scheduled for Oct. 1997 in Minneapolis, MN. Abstracts are due January 31, 1997.

- See the TwinSPIN home page for a link to PSQT web pages.


SUMMARY OF THE DECEMBER, 1996 PRESENTATION provided by Gail Bertossi

The PRESENTATION SUMMARY is divided into the following parts:

  • Topic
  • Presenter
  • Reference Material & URL
  • Presentation Slides
  • Discussion Summary

TOPIC: Return on Investment of Software Process Improvement

Mr. Burke Maxey led a discussion about measuring the return an organization achieves by software process improvement. He presented the published results of 5 companies. Discussion followed the presentatation.


DISCUSSION LEADER: Burke Maxey

Burke Maxey has over twenty years of experience in software development, consulting and management. He has participated in software process improvement efforts per ISO-9000, SEI CMM and TQM requirements. Burke has participated in software projects such as Joint STARS, F-14D, F-111, Tomahawk, and Iris communication system. Burke is currently employed at Rosemount Aerospace.


REFERENCE MATERIAL & URL

"Benefits of CMM Based Software Process Improvement: Initial Result" by Herbsleb, et al., CMU/SEI-94-TR-13, August 94, 64 pages.

URL: www.sei.cmu.edu/products/publications/94.reports/94.tr.013.html

SEI's ftp site (ftp.sei.cmu.edu) as the following filename: /pub/documents/94.reports/pdf/tr13_94.pdf
(Note that the report is a pdf document and requires Adobe Acrobat)

The Data and Analysis Center for Software (DACS) has published a report titled "A Business Case for Software Process Improvement." This report descirbes a (spreadsheet) model for analyzing costs and ROI from various types of SPI efforts and compares and contrasts cost benefits of Cleanroom Software Engineering, Formal Inspections, Reuse and full SPI.

The report is free and can be viewed through the DACS Web Page at http://www.dacs.com , under the Topic Area "Software Process Improvement," or a free hardcopy can be ordered by responding to this e-mail message.

The Microsoft Excel Spreadsheet developed in this research can be purchased from the DACS for $40. If interested, please contact:

Tom McGibbon (Director)
Data & Analysis Center for Software (DACS)
Kaman Sciences Corp.
775 Daedalian Drive
Rome, NY 13441-4909
Phone: (315) 334-4933
Fax: (315) 334-4965
e-mail: tmcgibbon@rome.kaman.com
DACS URL: http://www.dacs.com
The DACS is the DoD Software Information Clearinghouse, devoted to Software Technology.

PRESENTATION SLIDES
[commentary noted within square brackets]
Return on Investment (ROI) of Software Process Improvement (SPI)
Burke Maxey, Maxey Consulting Services, BFGoodrich/Rosemount Aerospace

INTRODUCTION

  • Provide some ROI Definition
  • Provide some examples of gains from SPI - Lead Discussion - Quiz
  • My SPI background
Multiple SEI CMM efforts
Multiple ISO 9000 implementations
Multiple ISO 9000 audits
Multiple SEPG memberships
  • Software Process Reverse Improvement (SPRI)

DEFINITION OF ROI

  • Accounting: Net Income divided by an average owner's equity over a yearly period
  • Webster's: Yield on an act that enhances value

ROI of SPI Examples

  • 5 Examples from five companies
  • All SEI CMM efforts
  • All previously published data
  • Companies are not identified until end of presentation
  • Quiz consists of Audience identifying the companies

Company One

Information Services Company
  • Local Connection
  • OS development group
  • 2 years CMM Level 1 to Level 2
Investment
  • SEPG group of 4-11 members [not full time]
  • SW training increased to 67 hrs/yr/person
  • 2% of development cost
Yield
  • Accumulation of data from past/present projects allows developers to make informed decisions
  • Savings of $3.5 M (US)/yr by better code and document inspection
  • Design Cycles longer; Code & test shorter [overall cycle shorter]
  • 7-10% reduction/yr of customer reported defects
ROI
  • 4 to 1

Company Two

Defense Contractor * 500 person division
* 1.5 years Level 2 to Level 3
* Ground Systems Division
Investment * 75 person months ($400,000)
* $45,000 assessment
Yield * Increased project predicatability
* Increased morale/working environment
* Earlier detection of defects
* Better at meeting schedule and cost
* First year benefit $2M
ROI 4.5 to 1

Company Three

Manufacturing Company
  • British division of U.S. company
  • 3 year SEI CMM effort
  • No levels specified
Investment
  • $170,000 US
Yield
  • 30% increase in productivity
  • Better at meeting schedules
  • Defects per KLOC went from .21 to .14
ROI
  • [improved to] 8.8 to 1 over 3.5 years

Company Four

Government military site
  • Oklahoma
  • 180 personnel
  • Three years Level 1 to Level 2
Investment
  • 5% of total labor
  • $1.07M
Yield
  • Direct savings of $4.79M
  • Decrease in rework & duplication of function
ROI
  • 4.5 to 1
  • could go up to 6.27 to 1

Company Five

Military Contractor
  • Software systems for equipment division
  • 400 software engineers
  • 5 years Level 1 to Level 3
Investment
  • About 15 people/year
  • About $1M/year for 5 years
Yield
  • Productivity increased by 2.3%
  • Software rework decreased to 11% from 41% of project cost
  • Higher employee morale
ROI
  • 7.7 to 1

Quiz - Who are these organizations?

  • One - BULL Honeywell
  • Two - Hughes
  • Three - Schlumberger
  • Four - OK City Air Logistics, Tinker AFB
  • Five - Raytheon

Quiz Grading

  • None correct: Try again
  • One correct: OK
  • Two correct: Good
  • Three correct: Terrific
  • Four correct: You must have worked there
  • Five correct: You read the same article.
DISCUSSION SUMMARY

ROI is always an estimate. It compares the old process with the new process.

ROI needs a baseline from the old process for comparison purposes. Many level 0 companies lack such a baseline.

ROI is calculated uniquely by each company based on their available data. There is no standard set of metrics.

ROI = (cost of benefits) divided by (cost of improvement)

Cost of improvement should include training, extra time, assessment costs, and personnel.

Cost of quality/benefits can be measured in terms of

  • cost to repair defects
  • cost of rework
  • cost of service desk staff
Defect Dollarization - Tim Olson

Tim Olson provided this method of defect dollarization.

  1. Count defects found at each stage (requirements, design, code, test, release)
  2. Ascertain the cost of fixing a defect based on the stage where cycle it is found.
  3. Multiply defect count * cost (for each stage).
  4. Add costs for all stages.
  5. Compare the defect dollars using the old process with the defect dollars using the new process. Calculate savings.
--------------------------

Tim Olson is giving a presentation on a Metrics Framework at the March SEPG conference in San Jose. If the SPIN is interested, he volunteered to give a dry run to the TwinSPIN before then. Maybe in March.