Comparing tools for acceptance testing of non-interactive calculators


TextTest

Fit/ColumnFixture

Test Persistence Format

plain text logs

table of inputs and expected outputs

Drives application via

command line, standard input, environment variables

API

Assertion mechanism

plain text file comparison

compares expected output values from table with those returned by test API

Code changes required to get started

plain text conversion of output
(can also log relevant behaviour)

creation of a test API

Languages supported

All

Java, .NET, Python, Perl, Smalltalk, C++

System resource usage testing

CPU time measured directly
Anything else can be logged
Can “save” exact or averaged results

Any measure can be returned via API methods No support for “saving”

Integration with other tools

Parallel testing on grid (SGE or LSF) Bug-reporting (Bugzilla)

Wiki (using FitNesse)