(ref.doc)dshaker 011093

From: [email protected] (Doug Shaker)
Newsgroups: comp.object,comp.lang.smalltalk,comp.lang.c++,comp.software.testing
Subject: FAQ - Testing Object-Oriented Programs
Date: 1 Oct 1993 14:43:55 GMT
Organization: Qualix Group Inc.
Distribution: world
Reply-To: [email protected]
NNTP-Posting-Host: compassion.qualix.com
Keywords: Testing OOP Quality Objects C++ Smalltalk


______________________________________________________________________

Testing of Object-Oriented Programming (TOOP) Resources Summary

Posted to comp.object, comp.lang.c++, comp.lang.smalltalk, and
comp.software.testing on 93.10.01.


What?
______________

This is a summary of resources on the Testing of Object-Oriented
Programming that have been mentioned to me over the net, in email,
or other means.  Sections include Written Material, Courses, and
Software.  It isn't really a FAQ list, mostly because people hardly
ever ask the questions.  However, it is a similar resource, so I
call it that in the subject line.

Suggestions for additions are eagerly encouraged.  As are copies
of relevant papers.


Who?
______________

I work for a Unix software house, Qualix Group, in the US.   Here is
my sig:
 - Doug Shaker
	voice:	415/572-0200
	fax:	415/572-1300
	email:	[email protected]
	mail:	Qualix Group
		1900 S. Norfolk St., #224
		San Mateo, CA 94403
I am NOT a researcher on the testing of object-oriented programming.
I just collate the stuff that is sent to me by people who REALLY know
something.  See the section "ACKs" at the end.

I just think it is important.

Why?
_______________

Why is this important? If classes are really to be reused in
confidence, they must be blatantly correct.  The classes must be easily
testable during initial evaluation by the client programmer.  They must
also be testable under different OS configurations, different compiler
optimizations, etc.  This means that testing modules must be
constructed in a way which is recognized as correct and the modules
must be shipped with the class libraries.

As soon as one major class library vendor starts to ship real test code
with their libraries, all of the other vendors will be forced, by
market pressure, to do so as well, or face market share erosion.  Think
about it.  If you had to recommend a class library to a committee that
was choosing a basis for the next five years of work, wouldn't you feel
safer with a class library that could be auto-tested in your
environment?


Written Material
_____________________

Berard, Edward.  Essays on Object-Oriented Software Engineering.  
	Prentice-Hall, Englewood Cliffs, NJ. $35.
	This book has two chapters on testing of object-oriented software, 
	focusing on how to do it.

Berard, Edward.  Project Management Handbook.  Must be purchased
	direct from Berard Software Engineering, Ltd., 902 Wind River
	Lane, Suite 203, Gaithersburg, Maryland 20878.  $225.
	The book focuses on the management of OOP porjects.  It
	includes one chapter on testing OO software and one chapter
	on quality assurance.

Cheatham Thomas J., and Lee Mellinger, "Testing Object-Oriented
	Software Systems",  Proceedings of the 18th ACM Annual Computer
	Science Conference, ACM, Inc., New York, NY, 1990, pp. 161-165.

Doong, Roong-Ko and Phyllis G. Frankl, "Case Studies on Testing 
	Object-Oriented Programs", Proceedings of the 4th Symposium on
	Testing, Analysis, and Verification (TAV4), 1991, ACM, Inc.,
	New York, NY, 1991, pp. 165-177.

Fiedler, Steven P., "Object-Oriented Unit Testing", Hewlett-Packard 
	Journal, April, 1989, pp. 69-74.

Firesmith, D.G., "Testing Object-Oriented Software", Proceedings 
	of 11th. TOOLS USA Conference, Santa Barbara, Aug 1993, pp 407-426.

Frankl, Phyllis G. and Roong-Ko Doong, "Tools for Testing 
	Object-Oriented Programs", Proceedings of the 8th Pacific
	Northwest Conference on Software Quality, 1990, pp. 309-324.
	One author can be reached at [email protected].

Graham, J.A., Drakeford, A.C.T., Turner, C.D. 1993. The Verification, 
	Validation and Testing of Object Oriented Systems, BT Technol
	J.  Vol 11, No 3. One author's email address is
	[email protected].

Harrold, Mary Jean, John D. McGregor, and Kevin J. Fitzpatrick, 
	"Incremental Testing of Object-Oriented Class Structures",
	International Conference on Software Engineering, May, 1992,
	ACM, Inc., pp. 68 - 80.

Hoffman, Daniel and Paul Strooper.  A Case Study in Class Testing.
	To be Presented at the IBM Center for Advanced Studies Fall
	Conference, October 1993, Toronto.  Email addresses for authors
	are [email protected] and [email protected].  Describes an
	approach to testing which the authors call Testgraphs.  An
	example is worked out in C++ which tests a commercial class.

Hoffman, D. M.  A CASE Study in Module Testing.  In Proc. Conf. Software
	Maintenance, pp. 100-105. IEEE Computer Society, October 1989.

Hoffman, D.M. and P.A. Strooper.  Graph-Based Class Testing.  In 
	7th Australian Software Engineering Conference (to appear), 1993.

Klimas, Edward "Quality Assurance Issues for Smalltalk Based Applications", 
	The Smalltalk Report, Vol. 1, No. 9, pp.3-7.  The author's
	email address is "[email protected]".

Leavens, G. T., "Modular Specification and Verification of 
	Object-Oriented Programs", IEEE Software, July 1991, pp. 72-80.

Love, Tom.  Object Lessons.  SIGS Books, 588 Broadway #604, New York, NY 
	10012. $49.
	This book eloquently elucidates the need for testing of object-
	oriented code and has a chapter on how it was done at Stepstone
	during the first release of their initial class library.

Murphy, G.C., Wong, P. 1992, Towards a Testing Methodology for 
	Object Oriented Systems, M.P.R Teltech Ltd. A poster at the
	Conference on Object Oriented Programming Systems, Languages
	and Applications ACM. Copies of this paper can be obtained
	through [email protected].

Murphy, G. and P. Wong.  Object-Oriented Systems Testing Methodlogy: An
	Overview.  Techical Report TR92-0656, MPR Teltech Ltd., October 
	1992.

Perry, D.E. and G.E. Kaiser, "Adequate Testing and Object-Oriented 
	Programming", Journal of Object-Oriented Programming, 
	2(5):13-19, Jan/Feb 1990.

Purchase, Jan A. and Russel L. Winder, "Debugging tools for 
	object-oriented programming", Journal of Object-Oriented 
	Programming, June, 1991, Vol. 4, No. 3, pp. 10 - 27.

Smith, M. D. and D. J. Robson, " A Framework for Testing Object-Oriented 
	Programs", JOOP, 5(3):45-53, June 1992.
	Describes ways in which the usual approach to software testing
	could be adapted for object-oriented software.
	This paper, or one with the same title and authors, is
	available by anonymouns ftp from vega.dur.ac.uk as
	"/pub/papers/foot.dvi".

Smith, M. D. and D. J. Robson, "Object-Oriented Programming - the 
	Problems of Validation",  Proceedings of the 6th International 
	Conference on Software Maintenance 1990, IEEE Computer Society 
	Press, Los Alamitos, CA., pp. 272-281.

Taylor, David. "A quality-first program for object technology", Object 
	Magazine, Vol. 2, No. 2, July-August 1992, pp17-18. SIGs
	Publications.  The article talks some about why testing is
	important for OOP and describes one quality program.

Turner, C. D. and D. J. Robson, "The Testing of Object-Oriented Programs",
	Technical Report TR-13/92, Computer Science Division, School of
	Engineering and Computer Sciences (SECS), University of Durham,
	England.
	Includes a survey of existing literature on testing of OO
	programs.  Testing of OOP is compared with traditional software
	testing.  A state-based approach is described.
	This paper is available by anonymous ftp from vega.dur.ac.uk in
	/pub/papers. Get "toop.ps.Z" for A4 paper and "toopus.ps.Z" for
	US letter paper formatting.

Turner, C. D. and D. J. Robson, "A Suite of Tools for the State-Based
	Testing of Object-Oriented Programs", Technical Report
	TR-14/92, Computer Science Division, School of Engineering and
	Computer Science (SECS), University of Durham, Durham,
	England.  Describes a series of tools for the generation and
	execution of test cases for OOP.  These tools assume a
	state-based testing approach.
	This paper is available by anonymous ftp from vega.dur.ac.uk in
	/pub/papers.  Get "tools.ps.Z" for A4 paper formatting or get
	"toolsus.ps.Z" for US letter formatting.

Turner, C. D. and D. J. Robson, "Guidance for the Testing of Object-
	Oriented Programs", Technical Report TR-2/93, Computer Science
	Division, School of Engineering and Computer Science (SECS),
	University of Durham, Durham, England.  Discusses different
	methods of making class declarations and the implications of
	those methods for testing.
	This paper is available by anonymous ftp from vega.dur.ac.uk in
	/pub/papers.  Get "guide.ps.Z" for A4 paper formatting or get
	"guideus.ps.Z" for US letter formatting.

Turner, C. D. and D. J. Robson, "State-Based Testing and Inheritance",
	Technical Report TR-1/93, Computer Science Division, School of
	Engineering and Computer Science (SECS), University of Durham,
	Durham, England.
	Discusses the implications of inheritance for testing,
	particularily incremental testing.
	This paper is available by anonymous ftp from vega.dur.ac.uk in
	/pub/papers.  Get toopinht.ps.Z" for A4 paper formatting or get
	"toopinhtus.ps.Z" for US letter formatting.

Wong, P. Automated Class Exerciser (ACE) User's Guide.  Technical
	Report TR92-0655, MPR Teltech Ltd., September 1992.
 
Courses
__________________________

Berard Software Engineering, Inc. teaches a seminar on Testing of
Object-Oriented Software (TOOS).  The next one scheduled that I know of
is November 8-12, in Washington.  Call 301-417-9884 for details.

Quality Fractals, Inc. has a course called "Testing Object-Oriented
Software".  Contact: 508-359-7273 (Box 337, Medfield, MA 02052).  The
course is taught by Shel Siegel of YESS!, Inc.  Contact: 916-944-1032.


Software
__________________________

There is a smalltalk class library in the Univ. of Illinois archives
which includes a simple Tester class written by Bruce Samuelson
([email protected]). It is a general superclass for application
specific classes that test non-interactive objects such as trees,
collections, or numbers. It is not suitable for testing user interface
components such as windows, cursors, or scroll bars. The filein
includes Tree classes, Tester itself, and subclasses of Tester that are
used to validate the Tree classes. For ParcPlace Smalltalk (ObjectWorks
4.1 and VisualWorks 1.0). To get it ftp the file
"/pub/st80_vw/TreeLW1.1" from st.cs.uiuc.edu.

IPL Ltd. (in the UK) has a testing tool called Cantata which allows for
testing C++, but as far as I am able to determine, it has no special
features for C++ testing.  From the product literature:
	Cantata allows testing to be performed in an intuitive way
	making the tool exceptionally easy to use and productive in
	operation. Cantata is suitable for testing software written in
	either C or C++.

	Cantata provides comprehensive facilities for all forms of
	dynamic testing, including: functional testing, structural
	testing, unit testing and integration testing. Cantata has been
	specifically designed to operate in both host and target
	systems and so allow full portability of tests between these
	environments.
For more information contact IPL:
	IPL Ltd.
	Eveleigh House, Grove Street, BA1 5LR
	UK
	(0225) 444888
	(0225) 444400 (FAX)

TestCenter from CenterLine will do coverage testing of C++ (and C)
code.  Also does some memory debugging and other QA functions.

MicroTech Pacific Research (mpr.ca) has a C++ class testing tool called
ACE (Automated Class Exerciser) which is available under non-disclosure
agreement.  It is not currently for sale.  If you are interested,
contact Paul Townsend, [email protected].

ACKs
___________________

Thanks to the following for helping assemble this list:
	Brian Marick, [email protected]
	Bruce Samuleson, [email protected]
	Daniel M. Hoffman, [email protected]
	Edward Klimas, [email protected]
	John Graham, [email protected]
	Jim Youlio, [email protected]
	Jeffery Brown, [email protected]
	Mary L. Schweizer, [email protected]
	Michael Einkauf, [email protected]
	Paul Townsend, [email protected]
	Phyllis G. Frankl, [email protected]
	Risto Hakli, [email protected]
	Russ Hopler, [email protected]
	Stephane Barbey, [email protected]
	Tony Reis, [email protected]
	Yawar Ali, [email protected]