Those companies which we audited which came out as being CMM Level 2 generally had very expansive processes and many "quality" metrics - rarely these metrics actually had something to do with the product being produced. Even more rarely those companies knew how to act upon those relevant metrics.
Now while I tend to focus on the methods, techniques and metrics of the actually job of engineering and dismiss process - or at least overbearing process - CMM does have its place but does require the audit to be carried out by not only process people but real, live engineers too.
CMM has been however the butt of many jokes - the most famous of which was started by Anthony Finkelstein in his famous 1992 paper: A Software Process Immaturity Model [1]. This was later taken up by Schorsch [2] in 1996 and expanded into the "The Capability Im-Maturity Model (CIMM)".
To understand these a quick explanation of the CMM levels is required. CMM rates a company's processes in terms of their maturity ranked on a scale of 0 to 5:
| Level | Name | Description of Maturity |
| 0 | Undefined | No audit or nothing at all |
| 1 | Initial | Chaotic, ad hoc and/or individual heroics |
| 2 | Repeatable | The process is at least documented sufficiently such that repeating the same steps may be attempted |
| 3 | Defined | The process is defined/confirmed as a standard business process |
| 4 | Managed | The process is quantitatively managed in accordance with agreed-upon metrics |
| 5 | Optimising | Process management includes deliberate process optimization/improvement |
Schorsch's expansion of this is:
| Level | Name | Description of Maturity |
| 0 | Negligent | The organization pays lip service, often with excessive fanfare, to implementing software engineering processes, but lacks the will to carry through the necessary effort. Whereas CMM level 1 assumes eventual success in producing software, CIMM level 0 organizations generally fail to produce any product, or do so by abandoning regular procedures in favor of crash programs. |
| -1 | Obstructive | Processes, however inappropriate and ineffective, are implemented with rigor and tend to obstruct work. Adherence to process is the measure of success in a Level -1 organization. Any actual creation of viable product is incidental. The quality of any product is not assessed, presumably on the assumption that if the proper process was followed, high quality is guaranteed. This is the most common level achieved by most organizations that pursue CMMI ratings. Paradoxically, Level -1 organizations believe fervently in following defined procedures, but lacking the will to measure the effectiveness of the procedures they rarely succeed at their basic task of creating software. Unfortunately, this behavior is inherent in the CMMI evaluation process. Since many government agencies will only award contracts over a certain dollar value to organizations that can pass a CMMI-3 or higher SCAMPI appraisal, management is more often than not willing to accept some inefficiencies in order to win these lucrative contracts. Government contracting models, in which organizations are paid not for the value of their products but by the number of hours spent building them, reward organizations for performing non-value-added activities related to CMMI compliance. Thus, government contractors with CMMI ratings will ultimately be more profitable than non-CMMI rated companies regardless of the quality of the software they produce. Whether CMMI processes provide any value in producing higher quality software is immaterial. |
| -2 | Contemptuous | While processes exist, they are routinely ignored by engineering staff and those charged with overseeing the processes are regarded with hostility. Measurements are fudged to make the organization look good. |
| -3 | Undermining | Not content with faking their own performance, undermining organizations routinely work to downplay and sabotage the efforts of rival organizations, especially those successfully implementing processes common to CMM level 2 and higher. This is worst where company policy causes departments to compete for scarce resources, which are allocated to the loudest advocates. |
NB: Finkelstein originally had levels 0 to -2 with the naming: 0 (foolish), -1 (stupid) and -2 (lunatic); the meanings are the same though.
Not content with faking their own performance, undermining organizations routinely work to downplay and sabotage the efforts of rival organizations, especially those successfully implementing processes common to CMM level 2 and higher. This is worst where company policy causes departments to compete for scarce resources, which are allocated to the loudest advocates.
After performing a number of review years ago this got me thinking at the time and I even wrote a short paper on this which will probably forever remain unpublished, namely as it is in some backup archive somewhere languishing on some slowly decaying magnetic media.
The CIMM (Capability Immaturity Model) is in the range [-3 to 5]:Integer. However I think it is more accurate to expand the CIMM levels into complex numbers, albeit integer ones. The two parts of a complex number are denoted real and imaginary - the meanings of those words fit nicely with auditing reality.
The eCIMM (Extended CIMM - my term!) level of a company can be expressed using both real and imaginary parts: the imaginary part, or what the company would like to believe, is what the auditors find out through the CMM audit, and the real part is actually what is happening inside the company. Just as CMM level 2 is the typically found, in the future this should be reported as something like eCIMM level -2+2i.
That is, the CMM auditors were presented with material showing that the company had spent much time on developing and documenting procesess and obtained a level 2 rating, but reality inside was that the processes exist but are routinely ignored and things fudged to make the organisation look good.
I guess that if we ever do find a CMM level 5 company, its eCIMM level would be 0+5i: a massive team of process people and one guy hacking (sorry, using agile methods) in the corner - and probably doing a good job of it too!
Now you could extend this by performing numerous audits and plotting the progress on and argand diagram - and you could probably come up with some interesting metrics and analysis of the distances between points - I guess it fits all the defintions of a metric and maybe might even be worth writing about.
- [1] A. Finkelstein, A Software Process Immaturity Model, SIGSOFT Software Engineering Notes, 1992.
- [2] Schorsch, "The Capability Im-Maturity Model (CIMM)", U.S. Air Force (CrossTalk Magazine), 1996.
- [3] Humphrey W.S.; Kitson D.H. & Kasse T.C. (1989); The State of Software Engineering Practice: a preliminary report; Proc. IEEE 11th International Conference on Software Engineering; pp 277-288, IEEE CS Press.
Postscript: at last I've written this down after years of not quite getting around to it...using complex numbers as CMM level indicators...
No comments:
Post a Comment