* NEWS: Document report card.

	* Makefile.am (clean-local): Add target.
	(clean-local-check): Add target; mark as PHONY.
	(commands_DATA): Add "report-card" scripts.
	(dist_man_MANS): Add dejagnu-report-card.1 and split.
	(DEJATOOL): Add "report-card" tool.
	(TESTSUITE_FILES): Add testsuite for "report-card" tool.

	* commands/report-card.awk: New command script.

	* doc/dejagnu.texi (Invoking dejagnu report card): New node.
	* doc/dejagnu-report-card.1: New man page.

	* testsuite/lib/bohman_ssd.exp: New file.
	* testsuite/lib/report-card.exp: New file.
	* testsuite/report-card.all/onetest.exp: New file.
	* testsuite/report-card.all/passes.exp: New file.

Signed-off-by: Ben Elliston <bje@gnu.org>

1 files changed, 146 insertions, 0 deletions

diff --git a/doc/dejagnu-report-card.1 b/doc/dejagnu-report-card.1
new file mode 100644
index 0000000..2e69fd0
--- /dev/null
+++ b/doc/dejagnu-report-card.1
@@ -0,0 +1,146 @@
+.\" Copyright (C) 2018  Free Software Foundation, Inc.
+.\" You may distribute this file under the terms of the GNU Free
+.\" Documentation License.
+.Dd December 31, 2018
+.Os GNU
+.Dt DEJAGNU-REPORT-CARD 1 URM
+.Sh NAME
+.Nm dejagnu\ report\ card
+.Nd summarize results from testing multiple tools
+.Sh SYNOPSIS
+.Nm dejagnu\ report\ card
+.Oo Ao Ar option Ac \*(Ba Ao Ar tool Ac \*(Ba Ao Ar file Ac Oc ...
+.Sh DESCRIPTION
+The
+.Nm
+command displays results from testing multiple tools in a tabular format.
+The produced table lists, for each tool (and if multiple passes were run,
+each pass) the number of tests passed, failed, unsupported, unresolved, and
+untested.  Tests that are expected to fail are counted in separate columns
+from tests expected to pass, but "known" failures and "expected" failures
+are summarized together.  If a test generated warnings or errors, a tag
+.Ql !W!
+or
+.Ql !E!
+is appended at the end of the relevant line.
+.Pp
+Aside from options, the argument list may include tool or file names.  The
+.Nm
+command prefers to read DejaGnu summary files and will translate names accordingly:
+.Bl -tag -width ".Pa *.sum"
+.It Pa *.sum
+Used as-is.
+.It Pa *.log
+Rewritten to
+.Pa *.sum
+with the same stem.
+.It Pa *.
+The string
+.Pa sum
+is appended to select a summary file.  This processing is done for
+convenience when using Readline file name completion in a shell, which will
+complete to the dot.
+.It Pa *
+Taken as a tool name;
+.Pa .sum
+is appended.
+.El
+.Sh OPTIONS
+.Bl -tag -width ".Fl v , -verbose"
+.It Fl v , -verbose
+Emit additional output describing the operation of
+.Nm
+itself.
+.El
+.Sh FILES
+The
+.Nm
+command produces its output by reading the summary files produced by
+DejaGnu and counting "PASS", "FAIL", etc.
+.Pp
+If no names are given as arguments, all files matching
+.Pa *.sum
+in the current directory are read.
+.Sh EXAMPLES
+.Ss A simple example from DejaGnu's own testsuite
+.Bd -literal
+$ dejagnu report card
+\             __________________________________________________
+\            /    PASS   FAIL  ?PASS  ?FAIL  UNSUP  UNRES UNTEST
+\            |--------------------------------------------------
+\  launcher  |      52      0      0      0      0      0      0
+libdejagnu  |       5      0      0      0      0      0      0
+\   runtest  |     135      0      0      0      0      0      0
+\            |--------------------------------------------------
+\            |     192      0      0      0      0      0      0
+\            \\__________________________________________________
+.Ed
+.Pp
+Three tools were tested, with a total of 192 tests, all expected to pass.
+In this example, all tests did pass, so all other columns are zero.  The
+.Ql ?PASS
+and
+.Ql ?FAIL
+columns count tests known or expected to fail that either unexpectedly
+passed or failed as expected.  The remaining three columns count the
+exceptional results for unsupported tests, unresolved tests and stub tests
+that simply declare themselves untested.
+.Pp
+.ne 16v
+.Ss The same example after tests were added for dejagnu-report-card
+.Bd -literal
+$ dejagnu report-card
+\                    __________________________________________________
+\                   /    PASS   FAIL  ?PASS  ?FAIL  UNSUP  UNRES UNTEST
+\                   |--------------------------------------------------
+\   launcher        |      52      0      0      0      0      0      0
+\ libdejagnu        |       5      0      0      0      0      0      0
+report-card / awk  |      36      0      0      0      0      0      0
+report-card / sh   |      36      0      0      0      0      0      0
+report-card / tcl  |      36      0      0      0      0      0      0
+\    runtest        |     135      0      0      0      0      0      0
+\                   |--------------------------------------------------
+\              awk  |      36      0      0      0      0      0      0
+\              sh   |      36      0      0      0      0      0      0
+\              tcl  |      36      0      0      0      0      0      0
+\                   |--------------------------------------------------
+\                   |     300      0      0      0      0      0      0
+\                   \\__________________________________________________
+.Ed
+.Pp
+The
+.Ql report-card
+tool has been added, with three passes, one for each implementation.  (The
+shell and Tcl implementations were later dropped to reduce future
+maintenance burden.)  As before, all tests passed as expected.  The
+interesting difference from the previous example is the use of DejaGnu's
+multipass testing feature and the additional per-pass summary lines added.
+For this example, only the
+.Ql report-card
+tool uses multipass testing, so each pass total is simply the count of
+tests for
+.Ql report-card
+instead of a distinct total.
+.Pp
+Also note that the command used to invoke
+.Nm
+is slightly different here.  The
+.Xr dejagnu 1
+launcher will also accept multiple words joined with dashes into a single
+argument.  This allows individual words in a command name to be separated
+with either dashes or spaces on the command line interchangeably.
+.Sh SEE ALSO
+.Xr dejagnu 1
+.Xr runtest 1
+.Pp
+The full documentation for DejaGnu is maintained as a Texinfo manual.  If the
+.Nm info
+program is properly installed at your site, the command
+.Li info dejagnu
+should give you access to the complete manual.
+.Sh AUTHORS
+.An Jacob Bachmeyer
+.\".Sh BUGS
+.\"  LocalWords:  Dt dejagnu URM Nm Ao Oo Oc DejaGnu Xr runtest DejaGnu's Bd Ql
+.\"  LocalWords:  testsuite UNSUP UNRES UNTEST libdejagnu Readline Ss tcl awk
+.\"  LocalWords:  ne multipass