build: script to filter known issues

This is is a proposal to have a system to filter the output of the
build (compilation, documentation, sanity check and runtime tests)
that eliminates known issues so that whoever sees the output of the
tree can note new issues being added without having to dive on
existing, known ones.

Most common user of this will be the continuous integration system, to
decide what is shown to gerrit as feedback to the user who submitted a
change.

The rationale behind having it in the tree is that if somebody submits
code that introduces a false positive (due to tool limitations) or as
an accepted (normally minor) issue to be fixed later, it can also
submit a "filter" for it without breaking CI.

For example, consider the documentation workaround in include/uart.h
(that will be reverted when this is done):

  diff --git a/include/uart.h b/include/uart.h
  index a30b211..178bd5e 100644
  --- a/include/uart.h
  +++ b/include/uart.h
  @@ -97,7 +97,7 @@ typedef void (*uart_irq_config_func_t)(struct device *port);
    * @param sys_clk_freq System clock frequency in Hz
    */
   struct uart_device_config {
  -       union __unnamed_workaround__ {
  +       union {
                  uint32_t port;
                  uint8_t *base;
                  uint32_t regs;

This introduces a harmless warning in the documentation compilation
process due to a limitation in the tools that will be fixed in future
releases. In the meantime, as they accumulate, it makes more difficult
for people to know if *they* introduced any other warnings (or
errors). The configuration in .known-issues/doc/uart.conf matches that
warning and filters it out (and only that), with enough regex glue to work
around subtle context changes (like line numbers).

The implementation is a Python script that can take the build output
and remove what is being told to ignore by a list of configuration
files, each of which contains a list of single/multiline Python
regular expressions.

Addition of said exceptions is caught by CI: it will trigger a
maintainer being included as a reviewer because the as directed by the
entry for the .known-issues in the MAINTAINERS file.

Change-Id: I7939e0726f2c505481592c3a7f5f40fa3e9c62fd
Signed-off-by: Inaky Perez-Gonzalez <inaky.perez-gonzalez@intel.com>

6 files changed, 135 insertions, 0 deletions

diff --git a/.known-issues/README b/.known-issues/README
new file mode 100644
index 000000000..ddd8f5a50
--- /dev/null
+++ b/.known-issues/README
@@ -0,0 +1,55 @@
+This directory contains configuration files to ignore errors found in
+the build and test process which are known to the developers and for
+now can be safely ignored.
+
+To use:
+
+ $ cd zephyr
+ $ make SOMETHING >& result
+ $ scripts/filter-known-issues.py result
+
+It is included in the source tree so if anyone has to submit anything
+that triggers some kind of error that is a false positive, it can
+include the "ignore me" file, properly documented.
+
+Each file can contain one or more multiline Python regular expressions
+(https://docs.python.org/2/library/re.html#regular-expression-syntax)
+that match an error message. Multiple regular expressions are
+separated by comment blocks (that start with #). Note that an empty
+line still is considered part of the multiline regular expression.
+
+For example
+
+---beginning---
+#
+# This testcase always fails, pending fix ZEP-1234
+#
+.*/tests/kernel/grumpy .* FAIL
+#
+# Documentation issue, masks:
+#
+# /home/e/inaky/z/kernel.git/doc/api/io_interfaces.rst:28: WARNING: Invalid definition: Expected identifier in nested name. [error at 19]
+#  struct dev_config::@65  dev_config::bits
+#  -------------------^
+#
+^(?P<filename>.+/doc/api/io_interfaces.rst):(?P<lineno>[0-9]+): WARNING: Invalid definition: Expected identifier in nested name. \[error at [0-9]+]
+^\s+struct dev_config::@[0-9]+  dev_config::bits.*
+^\s+-+\^
+---end---
+
+Note you want to:
+
+- use relateive paths; instead of
+  /home/me/mydir/zephyr/something/somewhere.c you will want
+  ^.*/something/somewhere.c (as they will depend on where it is being
+  built)
+
+- Replace line numbers with [0-9]+, as they will change
+
+- (?P<filename>[-._/\w]+/something/somewhere.c) saves the match on
+  that file path in a "variable" called 'filename' that later you can
+  match with (?P=filename) if you want to match multiple lines of the
+  same error message.
+
+Can get really twisted and interesting in terms of regexps; they are
+powerful, so start small :)
diff --git a/.known-issues/doc/bluetooth.conf b/.known-issues/doc/bluetooth.conf
new file mode 100644
index 000000000..d5e57d956
--- /dev/null
+++ b/.known-issues/doc/bluetooth.conf
@@ -0,0 +1,26 @@
+#
+# Bluetooth unnamed struct definition
+#
+# FIXME: all these should match the relative filename
+#
+^(?P<filename>[-._/\w]+/doc/api/bluetooth.rst):(?P<lineno>[0-9]+): WARNING: Invalid definition: Expected identifier in nested name. \[error at [0-9]+]$
+^[ \t]*$
+^[ \t]*\^$
+^(?P=filename):(?P=lineno): WARNING: Invalid definition: Expected identifier in nested name. \[error at [0-9]+]$
+^[ \t]*$
+^[ \t]*\^$
+^(?P=filename):(?P=lineno): WARNING: Invalid definition: Expected end of definition. \[error at [0-9]+]$
+^.*bt_conn_info.__unnamed__.*$
+^[- \t]*\^$
+#
+# Bluetooth GATT unnamed struct definition
+#
+^(?P<filename>[-._/\w]+/doc/api/bluetooth.rst):(?P<lineno>[0-9]+): WARNING: Invalid definition: Expected identifier in nested name. \[error at [0-9]+]
+^[ \t]*
+^[ \t]*\^
+^(?P=filename):(?P=lineno): WARNING: Invalid definition: Expected identifier in nested name. \[error at [0-9]+]
+^[ \t]*
+^[ \t]*\^
+^(?P=filename):(?P=lineno): WARNING: Invalid definition: Expected end of definition. \[error at [0-9]+]
+^.*bt_gatt_read_params.__unnamed__.*
+^[- \t]*\^
diff --git a/.known-issues/doc/header_footer.conf b/.known-issues/doc/header_footer.conf
new file mode 100644
index 000000000..a432d1c99
--- /dev/null
+++ b/.known-issues/doc/header_footer.conf
@@ -0,0 +1,23 @@
+#
+# Prints the PWD : FIXME, this is going to be removed
+#
+^pwd
+^.*/doc
+#
+# KERNELVERSION not being defined in local builds, kill that warning,
+# can ignore it
+#
+^.*/Kconfig.zephyr:[0-9]+: warning: The symbol KERNELVERSION references the non-existent environment variable KERNELVERSION.*
+#
+# Documentation generation, early message
+#
+^cd .* && doxygen doc/doxygen.config
+^srctree=.* SRCARCH=\w+ python scripts/genrest/genrest.py .*$
+# This cuts the sphinx build line; has to be separate because in the
+# middle, we have removed the KERNELVERSION one and a full regex won't match
+^sphinx-build -t \w+ -b html -d [-._/\w]+  -q \. .*
+#
+# Documentation generation, footer message
+#
+^[ \t]*
+^Build finished. The HTML pages are in [-._/\w]+
diff --git a/.known-issues/doc/sensor.conf b/.known-issues/doc/sensor.conf
new file mode 100644
index 000000000..1ce4359a3
--- /dev/null
+++ b/.known-issues/doc/sensor.conf
@@ -0,0 +1,12 @@
+#
+# Sensor value unnamed struct definition
+#
+^(?P<filename>[-._/\w]+/doc/api/io_interfaces.rst):(?P<lineno>[0-9]+): WARNING: Invalid definition: Expected identifier in nested name. \[error at [0-9]+]
+^[ \t]*
+^[ \t]*\^
+^(?P=filename):(?P=lineno): WARNING: Invalid definition: Expected identifier in nested name. \[error at [0-9]+]
+^[ \t]*
+^[ \t]*\^
+^(?P=filename):(?P=lineno): WARNING: Invalid definition: Expected end of definition. \[error at [0-9]+]
+^.*sensor_value.__unnamed__.*
+^[- \t]*\^
diff --git a/.known-issues/doc/uart.conf b/.known-issues/doc/uart.conf
new file mode 100644
index 000000000..1a6373cc2
--- /dev/null
+++ b/.known-issues/doc/uart.conf
@@ -0,0 +1,15 @@
+#
+# UART unnamed struct definition
+#
+^(?P<filename>[-._/\w]+/doc/api/io_interfaces.rst):(?P<lineno>[0-9]+): WARNING: Invalid definition: Expected identifier in nested name. \[error at [0-9]+]
+^[ \t]*
+^[ \t]*\^
+^(?P=filename):(?P=lineno): WARNING: Invalid definition: Expected identifier in nested name. \[error at [0-9]+]
+^[ \t]*
+^[ \t]*\^
+^(?P=filename):(?P=lineno): WARNING: Invalid definition: Expected identifier in nested name. \[error at [0-9]+]
+^[ \t]*
+^[ \t]*\^
+^(?P=filename):(?P=lineno): WARNING: Invalid definition: Expected end of definition. \[error at [0-9]+]
+^.*uart_device_config.__unnamed__.*
+^[- \t]*\^
diff --git a/.known-issues/make.conf b/.known-issues/make.conf
new file mode 100644
index 000000000..bbc14d7ba
--- /dev/null
+++ b/.known-issues/make.conf
@@ -0,0 +1,4 @@
+#
+# Make blurbs
+#
+^make: (Entering|Leaving) directory .*