





UCBTEST is a suite of programs for testing certain difficult cases of IEEE 754
floating-point arithmetic.  While most of the source code is NOT public
domain, the usage restrictions are modest; please read and observe the
copyright notices in the source code modules that have them.  Please report
any porting problems or logic errors to fdlibm-comments@sunpro.sun.com.  This
readme may be formatted with commands like "tbl readme | troff -ms".

     UCBTEST is a suite of programs for testing certain difficult cases of
IEEE 754 floating-point arithmetic.  Some of the difficult test cases are
obtained from number-theoretic algorithms developed by Turing Award winner
Prof. W. Kahan, Department of Electrical Engineering and Computer Science,
University of California, Berkeley, as part of ongoing research into test
methods for computer arithmetic.

     After Prof. Kahan presented a Floating-Point Indoctrination lecture
series at Sun Microsystems in 1988, some lecture attendees and former graduate
students of Prof. Kahan wrote programs to test the correctness of alleged IEEE
754 implementations of multiplication, division, and square root.  The suite
has subsequently expanded to include the following:

     ucbmultest - by number-theoretic means, generates multiplication test
cases that are exactly halfway and nearly halfway between representable
numbers.  The test algorithm is in ucbtest/mul.c.

     ucbdivtest - by number-theoretic means, generates division test cases
nearly halfway between representable numbers.  Exactly halfway cases don't
arise in division.  The test algorithm is in ucbtest/div.c.

     ucbsqrtest - by number-theoretic means, generates sqrt test cases nearly
halfway between representable numbers. Exactly halfway cases don't arise in
sqrt.  The test algorithm is in ucbtest/sqr.c.

     ucbpitest - attempts to determine a rational approximation A/B close to a
system's approximate value of J used in trigonometric argument reduction, and
tests whether the system's approximation is used consistently. ucbctest/pi.c
is a complete translation into C of ucbftest/pi.F, derived from the pirats
program.

     ucbpartest - attempts to determine general properties of computer arith-
metic by ostensibly portable means.   ucbftest/par.F and ucbctest/par.c are
complete translations from the original Basic program of Kahan.  After rework-
ing by various collaborators, ucbpartest's current pass/fail criterion
reflects one particular set of prejudices about x**0 and double rounding dur-
ing underflow; others may prefer different choices.

     ucbfindpitest - determines an approximation to a system's approximate
value of J used in trigonometric argument reduction, suitable for use in
ucbeef/local.h.

     ucbeefscan - a postprocessor that summarizes ucbeeftest's detailed out-
put.

     ucbeeftest - programs for testing sin, cos, atan, exp, expm1, log, and
log1p.  Random arguments test the functions for comparison to independently
computed values accurate to better than test function precision.  In UCBTEST,
the program is compiled and run separately for each function.  The testing
interface is thus somewhat different from the original form of the program
described in its README.  Note the usage restrictions set forth in that README
and source code.

     ucblibtest -  hex test vectors for library functions in single, double,
and quad precision.  Rounding precision control is not yet implemented, so
some x86 double test cases misleadingly appear to fail by one ulp, due to dou-
ble rounding.  x86 and m68k double extended are not supported.   Considerable
additional work remains to be done on this program.  The source codes provided
in UCBTEST may be helpful now, but are subject to ongoing improvement.

Pentium flaw

     An instance of the infamous Pentium FDIV flaw can be found in less than a
minute by compiling ucbdivtest in single precision with -DNTESTS=1000000 -DSP
etc.  That finding is unexpected since the program was not designed to seek
out such gross errors inherent in a particular division method, but rather
looks for least-significant-bit errors independent of division method.  Run-
ning in double or extended precision, ucbdivtest does not detect the Pentium
bug in a reasonable amount of time.

Porting

     All the UCBTEST programs are intended for users with a good understanding
of IEEE 754 arithmetic and C.  Some sample *.sh scripts in ucbREADME/ compile
and run all the programs, for three floating-point types: IEEE 754 single,
double, and double-extended (including quad).  The test programs are designed
to terminate with non-zero exit status in case they find an error, unless com-
piled with -DNO_FAIL, which causes zero exit status even in error cases.  At
the end, the scripts summarize run-time failures indicated by UCBFAIL in the
output or by the existence of incomplete *.output.tmp files.  The criterion
for ucblibtest is that all test vectors succeed; the criterion for ucbeeftest
is that no monotonicity or symmetry failures occur and that the worst case
errors are less than one ulp.  These summaries don't catch compile and link
errors - those errors terminate Makefiles and scripts unless make -k is used.

     ucbmultest, ucbdivtest, ucbsqrtest, and ucblibtest were developed on
SPARCstations running SunOS 4 or SunOS 5, and on x86 PC's running SunOS 5.
ucbdivtest and ucbsqrtest may be ported to other RISC/Unix systems with little
difficulty.   Systems with extended precision, non-Unix systems, and ucbmul-
test require more effort to provide the equivalents of the Sun functions
ieee_flags etc.

     On SPARCstations, "int" and "long" and Fortran "INTEGER" are 32 bits
wide.  On DOS systems, "int" is usually 16 and "long" 32.  On some supercom-
puters, "int" and "long" might be 64.  In those cases, these definitions in
libctest and ucbtest may require modification:

libctest.h: #define INTEGERTYPE int
libctest.h: #define INTEGER INTEGERTYPE *

ucbtest.h: #define FINT    int             /* C equivalent of Fortran integer */
ucbtest.h: #define INT32   int             /* 32 bit int */
ucbtest.h: #define UINT32  unsigned int    /* 32 bit unsigned */


     Some Fortran compilers append underscores to subprogram identifiers - in
order to avoid name space conflicts when C and Fortran programs are mixed -
and some do not.   Since UCBTEST depends on that underscore differentiation,
aix.sh for instance specifies "-DUNDERSCORE=_" in $cppopts, and
libftest/libftest.h and ucbftest/ucbftest.h provide definitions like

#if defined(__hpux) || defined(_AIX)
#define UNDERSCORE _
#else
#define UNDERSCORE
#endif

HPUX and AIX require the explicit underscore because they don't provide it by
default.

Misapplication

     Misapplication often causes these programs to fail, reporting hardware or
software bugs where none exist.  If misapplied - especially after porting to a
new environment for the first time - they may report arithmetic "errors" when
nothing is amiss.   Nor do "error-free" results prove correct arithmetic,
although they may "lend artistic verisimilitude to an otherwise bald and
unconvincing" proof.  Common misapplications include:

precision
     On extended-precision systems such as x86 and 68k, the extended-precision
     precision control must be set to the precision of the type of floating-
     point variable being tested.  On Sun systems,
     ieee_flags("set","precision",...) is useful.

prototypes
     Traditional C and ANSI-C differ in their treatment of float function
     arguments and function results.

traditional C expression evaluation
     Traditional C may evaluate expressions involving floats by promoting them
     first to double, causing two rounding errors where UCBTEST expects one.
     Compiling with "-fsingle" is helpful with /bin/cc on SunOS 4.

optimizers
     Optimizing compilers that allocate variables to extended-precision regis-
     ters and fail to round them to storage precision, even in response to
     source-code assignments, may cause problems.   Certain variables have
     been declared "volatile" in hope of avoiding those problems.  Particu-
     larly on PC's, compilers seem to have trouble correctly optimizing code
     and libraries to work with rounding precision set to single instead of
     extended.  Perhaps more "volatile" declarations are required.   On GCC,
     -ffloat-store sometimes helps.

sqrt functions
     C compilers usually don't convert calls to sqrtf, sqrt, or sqrtl to
     inline machine sqrt instructions by default. Thus testing those instruc-
     tions, rather than the subroutine library, may require extra steps such
     as coding in assembly language.    Sun compilers invoked with "-xlibmil"
     provide the necessary inlining to test SPARC or x86 sqrt instructions.

Source code

     Some compile-time flags used to configure the tests include:

         -DNTESTS     number of test cases to try
         -DDEBUG      turn on debugging output
         -DSUN_IEEE   ieee_flags, ieee_retrospective available
         -DSUN_MATH   sunmath.h and libsunmath.a are available
         -DSUN_LIBS   Fortran-callable versions available for
                      Sun libm and libsunmath functions
         -DNO_FUNCF   no ANSI-C reserved functions sqrtf, expf, etc.
         -DNO_FAIL    do not abort on certain errors
         -DSP         test IEEE single precision
         -DDP         test IEEE double precision
         -DQP         test IEEE extended precision


     Several of the programs are set up so that the C infrastructure may be
used to test the arithmetic and libraries of any language.  Sample wrappers
are provided for C and Fortran.  Outer main programs are in ucbctest/ and
ucbftest/; they use inner main programs and utility functions from ucbtest/ to
invoke the test functions in libctest/ or libftest/.

     Makefiles are provided in the ucbctest/, ucbftest/, libctest/, and
libftest/ directories. Disable .SILENT: targets for more debugging output;
enable them for less.  The Makefiles avoid infinite loops in ucblibtest, but
aux.sh gets into infinite loops in ucbpartest and ucbmultest in single preci-
sion.   In order to continue UCBTEST, such looping processes need to be ter-
minated manually with a kill command.

     Sample *.sh scripts for invoking the Makefiles are in ucbREADME/.   The
scripts define -NTESTS=1000000; reduce that to 1000 to simplify debugging;
increase it to one billion for a more comprehensive test.
ucb{mul,div,sqr,eef}test have been adjusted so their run times are approxi-
mately comparable for comparable NTESTS.  ucb{pi,par,findpi,lib}test run times
do not depend on NTESTS.

     The scripts define which precisions and which languages are to be tested,
how each language is to be compiled, and whether (via make -k)
script/compile/link errors terminate execution.  To create a script for a new
platform, start by copying one of the existing scripts for a similar system,
or start with generic.sh and modify it according to the suggestions in its
comments until it works acceptably.

     Generally speaking, the source code uses -DSP, -DDP, and -DQP to distin-
guish single, double, and extended precision.  -DQP refers flexibly to
double-extended on x86 and quadruple on SPARC on all the tests except ucblib-
test, where the inputs and output test vectors are specified in hex, so -DQP
means SPARC quadruple precision, and double-extended isn't supported.  The
programs start out by setting extended rounding precision on x86 to match that
of the nominal precision (SP, DP, QP), but because some of the libm functions
misbehave when invoked that way, ucb{pi,findpi,eef}test prevent that by res-
toring the default_precision.

     The source code file ucbtest/fpce.S is intended to provide a convenient
place to integrate assembly-language implementations of the functions to set
IEEE rounding direction and precision.  sunos5.gcc.sh depends on those to pro-
vide a substitute for ieee_flags in Sun's unbundled libsunmath.a.  Unfor-
tunately in Solaris 2, there's no convenient way to set const _lib_version,
defined in <math.h>, to -1 in order to get the same effect as Sun's -xlibmieee
option; this causes some ucblibtest vectors to fail with gcc.    That can be
remedied by reconfiguring GCC to link with values-Xi.o instead of values-X?.o
(see config/svr4.h) where values-Xi.c is:

#include <math.h>
const enum version _lib_version = libm_ieee;


Directory information

     ucbctest/ and ucbftest/ contain the main programs for each test, in C and
Fortran respectively.

     libctest/ and libftest/ contain the test modules which invoke each func-
tion to be tested, in C and Fortran respectively.   Each function is in a
separate module so that compilation failure on one will not affect other
tests.   On systems which lack some common library functions, it is necessary
to improvise; see for instance libctest/testexpm1*.c.

     ucbtest/ contains the C logic for the programs that can be invoked from
main programs in C or Fortran.

     ucbeef/ contains various source files invoked by ucbeeftest and ucbfind-
pitest.   See the separate README.

     ucblib/ contains the input files for ucblibtest.

     ucbREADME/ contains this readme and also readme.about.testing that
discusses the limitations of various testing approaches.  The various *.sh
sample test scripts are here, along with logs of typical executions.

     The sc301.sh script for Sun's SC3.0.1 compilers on SunOS 5.3 passes the
largest number of successful tests of any of these sample scripts, not
surprisingly because predecessors of UCBTEST have been part of Sun's compiler
test regime for years.

     ucbDOS contains Warren Ferguson's adapted source codes and compiled exe-
cutables for MSDOS.   See the separate readme.

Results Obtained During Development

     The  following  results  are  recorded  in  *.log  files  obtained   with
corresponding  *.sh  scripts.   In many cases the hardware, operating systems,
compilers, and libraries may not be current products, and the compilation  op-
tions may not be optimal. The sample *.sh scripts are only starting points and
may require modification when hardware, operating systems, compilers,  or  li-
braries change.  Many of the anomalies in the logs have not been investigated;
some may represent problems with the hardware, operating  systems,  compilers,
or  libraries;  problems  with  the UCBTEST port may underlie other anomalies,
corresponding to the various types of "misapplication" discussed above.   Con-
tributions of improved scripts and log files are welcome.

     The table indicates the fraction of "UCBPASS tests" / "total tests"  pos-
sible  with the tested languages and precisions, and when available, the total
time in minutes, either real or user+sys, for the script  to  create  the  log
file.  In  most cases other jobs were also running on the test systems, so the
times are very approximate.  Some of the log files were produced by early ver-
sions of the scripts that don't report as much information as the current ver-
sions.

                           RISC systems with 113-bit extended or none
8D'l |2304u 0'D'l |2304u 0'
 .sh              hardware            c                                          UCBPASS
 .log             op sys              fortran                                    minutes
8D'l |2304u 0'D'l |2304u 0'

sc301             SS10/41             3.0.1 cc -O -xlibmil -xlibmieee ...        77/84
                                      ... -xcg92 -fsingle -lsunmath -lm
sc301             SunOS 5.3           3.0.1 f77 -O -xlibmil -xcg92               210 user+sys

sunos4            SPARCclassic        /bin/cc -O /usr/lib/libm.il -fsingle -lm   56/56
sunos4            SunOS 4.1.3U1       1.0 f77 -O /usr/lib/libm.il                170 user+sys

apogee30          SPARC 2000, 60MHz   3.0 apcc -O -cg92 -Dlint -lm               38/56
apogee30          SunOS 5.4           3.0 apf77 -O -cg92                         49 user+sys

hpux              HP 9000/712         /bin/c89 -O -lm -DNTESTS=10000             34/70
hpux              HP-UX 9.05          /usr/bin/f77 -O -DNTESTS=10000             60 real

irix              SGI ?               /usr/bin/cc -O -lm                         24/28
irix              IRIX 5.2            -                                          60 real

mips.ultrix.gcc   DEC mips            gcc -O -lm                                 21/28
mips.ultrix.gcc   Ultrix 4.1          -                                          ?

alpha.osf.gcc     DEC alpha           gcc -O -ffloat-store -lm                   11/14
alpha.osf.gcc     OSF 3.0             -                                          ?


                                       CISC systems with 64-bit extended
8D'l |2688u 0'D'l |2688u 0'
 .sh                hardware            c                                                       UCBPASS
 .log               op sys              fortran                                                 minutes
8D'l |2688u 0'D'l |2688u 0'

pc201               Pentium 90 fixed    2.0.1 cc -O sqrt.il -xlibmieee -xpentium -fsingle -lm   54/70
pc201.fix           SunOS 5.4           2.0.1 f77 -O sqrt.il -pentium                           89 user+sys

pc201               Pentium 90 flawed   2.0.1 cc -O sqrt.il -xlibmieee -xpentium -fsingle -lm   52/70
pc201.flaw          SunOS 5.4           2.0.1 f77 -O sqrt.il -pentium                           88 user+sys

linux               486 66DX2           /usr/bin/cc -O -ffloat-store -lm                        44/56
linux               slackware 1.1.59    /usr/bin/f77 -O                                         170 user+sys

sunos3.fpa          Sun-3/260+FPA       /bin/cc -O -ffpa /usr/lib/ffpa.il -fsingle -lm          41/56
sunos3.fpa          SunOS 3.5.2         /usr/bin/f77 -O -ffpa /usr/lib/ffpa.il                  1300 user+sys

aux                 Apple Quadra 800    /bin/cc -O -lm                                          40/56
aux                 A/UX 3.0.2          /usr/bin/f77 -O                                         440 real

sunos5.gcc          Pentium 90 fixed    2.6.3 gcc -O -ffloat-store -lfdm                        24/28
sunos5.gcc          SunOS 5.4           -                                                       43 real

sunos5.fdlibm.gcc   Pentium 90 fixed    2.6.3 gcc -O -ffloat-store -lfdm                        24/28
sunos5.fdlibm.gcc   SunOS 5.4           -                                                       32 user+sys

m68.hp.netbsd1.0a   HP m68k             gcc -O -D_POSIX_SOURCE                                  24/28
m68.hp.netbsd1.0a   BSD 1.0a            -                                                       ?

sunos3              Sun-3/260           /bin/cc -O -f68881 /usr/lib/f68881.il -fsingle -lm      ?/56
sunos3              SunOS 3.5.2         /usr/bin/f77 -O -f68881 /usr/lib/f68881.il              ?


Source Code Authors; Translators, Porters, Maintainers

     ucbdivtest - K.-C. Ng; M. Alemi, S. Ma, D. Priest.

     ucbeeftest - Z. A. Liu inspired in part by W.  J.  Cody  and  W.  Waite's
ELEFUNT.

     ucbfindpitest - Z. A. Liu.

     ucblibtest - K.-C. Ng.

     ucbmultest - B. Toy; M. Alemi, S. Ma, M. Mueller, K.-C. Ng, D. Priest.

     ucbpartest - W. Kahan inspired in part by W. J. Cody's MACHAR;  D.  Feen-
berg, D. Gay, R. Karpkinski, T. Quarles, T. Sumner, G. Taylor, B. Wichmann.

     ucbpitest - W. Kahan; D. Priest.

     ucbsqrtest - K.-C. Ng; M. Alemi, S. Ma, D. Priest.

     Z. A. Liu generously permitted distribution, as part  of  UCBTEST,  of  a
version of his Berkeley Elementary Function tests, according to the conditions
set forth in the comments of those source codes.

     Sun Microsystems generously permitted distribution, as part  of  UCBTEST,
of versions of certain source codes and test vectors developed at Sun, accord-
ing to the conditions set forth in the header comments of those  source  codes
and test vectors.

Porting and Quality Acknowledgements

     David Gay improved portability to non-Solaris workstations running  vari-
ous proprietary compilers. Stephen Moshier improved portability to non-Solaris
workstations running GCC.  Warren Ferguson improved portability to  MSDOS  and
contributed the ucbDOS directory.  Intel lent systems equipped with flawed and
unflawed Pentiums for testing purposes.

Problems

     Please report bugs and portability  enhancements  for  other  systems  to
fdlibm-comments@sunpro.sun.com.

Futures

     Enhancements planned to be incorporated some day:

*    Ng's libm test vectors - ucblibtest - are still in the process of conver-
     sion  to  the  UCBTEST format, and the program's tests for rounding modes
     and exceptions are not yet working portably.

*    Kahan's tests for SRT-PLA-style dividers.

*    Coonen's IEEE Test Vectors program and data.

*    Hough's tests for correctly-rounded base conversion.
























































































































































































