(ref.doc)olavi 280693

Newsgroups: comp.software.testing,comp.software-eng
From: [email protected] (Poutanen Olavi)
Subject: Re: coverage testing tools
Nntp-Posting-Host: kaarne.cs.tut.fi
Organization: Testwell
Distribution: comp
Date: Mon, 28 Jun 1993 11:16:00 GMT

In article <[email protected]> [email protected] (Tahsin Alam [temp]) writes:

>Hi:
>
>I am looking for tools that will allow one to measure the coverage
>(which routines were called, which binary branches were taken, etc.)
>achieved by running test suites. I already know about ...

WE ALSO ...

Testwell has some testing tools for Ada, C and C++ languages. Besides
test coverage tools we also have some test harnessing tools for unit
and unit integration level testing enabling the testing against the
intended functionality. Namely, even 100% test coverage alone does not
guarantee the correct behaviour. I hope that the short tool descriptions
below interest the audience of this newsgroup.

For Ada we have:

TBGEN   Test Bed Generator for Ada. This is a specification-based (black-
        box) unit testing (and unit integration testing) tool for Ada.
        Here a unit, or a module, is normally Ada's package. It has a call
        interface. Your need is to test it with various call sequences
        and parameter values and observe/check the results.

        TBGEN generates the testing environment, the test bed, for you.
        Also stubs can be generated if needed. The test bed allows both
        interactive and script based testing. A test session might look:

           MYBED> i : integer := 5 -- Ada-like command language
           MYBED> arr : pack.int_arr(1 .. 3) := (10, 20, 30)
           MYBED> -- Next is a test call. Out-parameters (or exception
           MYBED> -- raised) shown afterwards, i.e. make the tests visible
           MYBED> pack.proc(data => 100, p2 => i, buff => arr)
                  P2 (out) = 95
                  BUFF (out) = (110, 120, 130)
           MYBED> -- automatic result checking (normally in scripts)
           MYBED> assert not exception and
           MYBED+        i = 95 and arr(1) = 110 and arr(3) = 130
           MYBEB> ...

        The scripts may be nested, can have loops and ifs. Trace is written.
        The testing is standardized, visible, "easy" (as what comes to
        arranging the environment), automated, documented, measurable,...

TCMON   Test Coverage Monitor for Ada. Instrumentation based tool. Gives
        you (depending what options you take) statement coverage,
        condition coverage, subcondition coverage, subprogram execution
        timing (your own cost function can be used), timing of user defined
        events. TCMON is a simple, easy-to-use and efficient tool to ensure
        thorought testing and to complement the specification-based testing
        approach.

For C and C++ we have:

CTB     C Test Bed. A similar tool for C (sorry not for C++) that TBGEN
        is for Ada (see above). Written in portable ANSI C, assumes only
        ANSI C -- well, in practice we recognize also some dialects.
        Based on C header files generates (no user programming is needed!)
        very powerful unit test (and unit integration test) environment,
        the test bed. C-like command language. C.f. TBGEN above.

CTC++   Test Coverage Analyzer for C/C++. Instrumentation based tool.
        Gives you Function Coverage (the most lightweight instrumentation:
        have all functions been called), Decision Coverage (decisions and
        other control transfers), Interface Coverage (applies to C++
        classes only: how thoroughly the class interface has been exercized
        at each inheritance level), Function Execution Timing (you can supply
        your own cost function), and measuring of user-defined events.
        CTC++ is a simple, easy-to-use and efficient tool. Can be used
        together with CTB, too.

C++ Test Aider. This little pearl is a test harnessing tool for C++ classes.
        Its goals are quite the same as with TBGEN for Ada and CTB for C but
        the implementation technique is different. The tool provides you a 
        special Test Aider Language for writing the test driver on your
        C++ code to be tested. So, when in TBGEN and CTB test beds the test
        cases come from the terminal or from textual script files, here you
        have them hard-coded in your test driver. However, some of the
        important properties are preserved: trace writing of Test Aider
        Program execution (test progress is made visible and documented),
        assertions can be used, testing practices can be standardized,
        testing is repeatable. The private part of a class can be accessed
        at testing time.

Besides these testing tools Testwell markets a complexity measures tool
(McCabe, Halstead, line-of-code metrics for C/C++) and style guides for
C and C++ languages.

I hope the above gave you some true information of Testwell's product
offering. For more information, please contact.

Olavi Poutanen
Managing Director

Testwell Oy
Kanslerinkatu 8, SF-33720 Tampere, Finland
Tel. +358-31-316 5464, Fax. +3658-31-183 311
(Can be reached also with the following email: [email protected])