This Article 
 Bibliographic References 
 Add to: 
Prototyping a Process Monitoring Experiment
October 1994 (vol. 20 no. 10)
pp. 774-784

Features are often the basic unit of development for a very large software system and represent long-term efforts, spanning up to several years from inception to actual use. Developing an experiment to monitor (by means of sampling) such lengthy processes requires a great deal of care in order to minimize casts and to maximize benefits. Just as prototyping is often a necessary auxiliary step in a large-scale, long-term development effort, so, too, is prototyping a necessary step in the development of a large-scale, long-term process monitoring experiment. Therefore, we have prototyped our experiment using a representative process and reconstructed data from a large and rich feature development. This approach has yielded three interesting sets of results. First, we reconstructed a 30-month time diary for the lead engineer of a feature composed of both hardware and software. These data represent the daily state (where the lead engineer spent the majority of his time) for a complete cycle of the development process. Second, we found that we needed to modify our experimental design. Our initial set of states did not represent the data as well as we had hoped. This is exemplified by the fact that the "Other" category is too large. Finally, the data provide evidence for both a waterfall view and an interactive, cyclic view of software development. We conclude that the prototyping effort is a necessary part of developing and installing any large-scale process monitoring experiment.

[1] T. J. Allen,Managing the Flow of Technology. Cambridge, MA: MIT, 1977.
[2] N. S. Barghouti and G. E. Kaiser, "Modeling concurrency in rule-based development environments,"IEEE Expert, pp. 15-27, Dec. 1990.
[3] V. R. Basili and D. M. Weiss, "A methodology for collecting valid software engineering data,"IEEE Trans. Software Eng., vol. SE-10, no. 6, pp. 728-738, Nov. 1984.
[4] V.R. Basili and H.D. Rombach, "The Tame Project: Towards Improvement-Oriented Software Environments,"IEEE Trans. Software Eng., Vol. SE-14, No. 6, June 1988, pp. 758-773.
[5] V. Basili et. al, "The Software Engineering Laboratory - An Operational Experience Factory,"Proc. Int'l Conf. software Eng., CS Press, Los Alamitos, Calif., 1992.
[6] M. Dowson, Ed.,Iteration in the Software Process: Proc. 3rd Int. Software Process Workshop, Breckenridge, CO, USA, Nov. 1986.
[7] M. Dowson, Ed.,Proc. 1st Int. Conf. Software Process: Manufacturing Complex Syst., Redondo Beach, CA, USA, Oct. 1991.
[8] R. Guindon, "Designing the design process: Exploiting opportunistic thoughts,"Human Comput. Interaction, vol. 5, pp. 305-344, 1990.
[9] Harel, D. et al., "Statemate: A Working Environment for the Development of Complex Reactive Systems," inProc. 10th IEEE Int'l Conf. Software Eng., Apr. 1988, CS Press, Los Alamitos, Calif., Order No. 849.
[10] W. S. Humphrey,Managing the Software Process. Reading, MA: Addison-Wesley, 1989.
[11] C. M. Judd, E. R. Smith, and L. H. Kidder,Research Methods in Social Relations, 6th ed. Orlando, FL: Harcourt Brace Jovanovich, 1991.
[12] T. Katayama, Ed.,Support for the Software Process: Proc. 6th Int. Software Process Workshop, Hakodate, Japan, Oct. 1990.
[13] M. I. Kellner, "Software Process Modeling Support for Management Planning and Control," inProc. 1st Int. Conf. Software Process: Manufacturing Complex Syst., 1991, pp. 8-28.
[14] G. A. Miller, "The magic number seven, plus or minus two: Some limits on our capacity for processing information,"Psychology Rev., vol. 63, no. 2, pp. 81-97, Mar. 1956.
[15] A. Nozari and W. Whitt, "Estimating average production intervals using inventory measurements: Little's Law for partially observable processes,"Operations Res., vol. 36, p. 308, Mar.-Apr. 1988.
[16] L. J. Osterweil, Ed.,Proc. 1st Int. Conf. Software Process: Continuous Software Process Improvement, Berlin, Germany, Feb. 1993.
[17] D. E. Perry,Experience with Software Process Models: Proc. 5th Int. Software Process Workshop, Kennebunkport, ME, USA, Oct. 1989.
[18] D. E. Perry, "Policy-directed coordination and cooperation," inProc. 7th Int. Software Process Workshop, Yountville, CA, USA, Oct. 1991.
[19] D. E. Perry, N. A. Staudenmayer, and L. G. Votta, "People, organizations, and process improvement,"IEEE Software, vol. 11, pp. 36-45, July 1994.
[20] C. Potts, Ed.,Proc. Software Process Workshop, Egham, Surrey, England, UK, Feb. 1984.
[21] Process Virtual Machine Working Group, DARPA Prototech Community, personal commun., 1991-1992.
[22] P. Mi and W. Scacchi, "Modeling articulation work in software engineering processes," inProc. 1st Int. Conf. Software Process: Manufacturing Complex Syst., 1991, pp. 188-201.
[23] R. W. Selby, A. A. Porter, D. C. Schmidt, and J. Berney, "Metric-driven analysis and feedback systems for enabling empirically guided software development,"Proc. 13th Int. Conf. Software Eng., 1991, pp. 288-298.
[24] R.N. Taylor, R. W. Selby, M. Young, F. C. BeIz, L. A. Clarke, J. C. Wileden, L. Osterweil, and A. L. Wolf, "Foundations for the Arcadia environment architecture," inACM SIGSOFT '88: Third Symp. Software Development Environments, Boston, MA, Nov. 1988, pp. 1-13; see alsoSIGSOFT Software Eng. Notes, vol. 13, no. 5, Nov. 1988; see alsoSIGPLAN Notices, vol. 24, no. 2, Feb. 1989.
[25] I. Thomas, Ed.,Proc. 7th Int. Software Process Workshop, Yountville, CA, USA, Oct. 1991.
[26] C. Tully, Ed., "Representing and enacting the software process," inProc. 4th Int. Software Process Workshop, ACM SIGSOFT(Moretonhampstead, Devon, UK), 1988, vol. 14, no. 4.
[27] J. C. Wileden and M. Dowson, Eds.,Software Process and Software Environments: Proc. 2nd Int. Software Process Workshop, Coto de Caza, CA, USA, Mar. 1985, inACM SIGSOFT Software Eng. Notes, vol. 11, Aug. 1986.
[28] A. Wolf and D. Rosenblum. "A Study in Software Process Data Capture and Analysis,"Proc 2nd Int'l Conf. Software Process, IEEE CS Press, Los Alamitos. Calif., 1993, pp. 115-124.

Index Terms:
software prototyping; process computer control; computerised monitoring; process monitoring experiment; prototyping; very large software system; large-scale long-term development; long-term process monitoring; hardware; software; experimental design; waterfall view; interactive cyclic view; software development; large-scale process monitoring experiment
M.G. Bradac, D.E. Perry, L.G. Votta, "Prototyping a Process Monitoring Experiment," IEEE Transactions on Software Engineering, vol. 20, no. 10, pp. 774-784, Oct. 1994, doi:10.1109/32.328994
Usage of this product signifies your acceptance of the Terms of Use.