1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
|
.. index:: user notifications, notifications
.. _notifications:
User notifications in LAVA
##########################
Users can receive notifications containing information about submitted test
jobs and the accompanying results by using settings in job definition yaml.
Users can also compare the job results within notifications against previous
results which satisfy certain conditions.
The basic setup of the ``notify`` block in job definitions will have
**criteria**, **verbosity**, **recipients** and **compare** blocks.
* **Criteria** tells the system when the notifications should be sent.
* **verbosity** will tell the system how detailed the email notification should
be.
* **Criteria** can be set by job status (running, complete, incomplete,
canceled) and the type (regression, progression). There is a special status of
'finished', which will match any of (complete, incomplete, canceled).
The basic **email** template includes job details, device details and results,
based on the level of **verbosity** (``verbose``, ``quiet``, ``status-only``).
verbose level job information will only be included if the job finished as
complete or incomplete, not when the job was cancelled.
Notification recipients
=======================
The method and destination of each notification can be set for each recipient.
Currently, two notification methods are supported, **email** and **IRC**.
Recipients can be specified using the LAVA username or in full.
If the **recipients** section is omitted in the notify block, the system will
send an email to the job submitter only, provided the **criteria** is satisfied,
and there is no **callbacks** section.
Notification callbacks
======================
In addition to sending email and IRC messages to **recipients**, the system can
send multiple URL callback actions. This will do a GET or POST request to the
specified URL in the **callbacks** subsection. This can be used to trigger some
action remotely. If a callback uses the POST request, the system will attach job data as described below.
The **callbacks** section supports list of the following options:
* **url** The URL for the request. This also supports field value
substitution, i.e. in http://my.remote.site/{ID}/{STATUS} **id** and
**status** will be replaced with corresponding values from the job.
* **method** GET or POST
* **token** This option is used to supply the API token of the
authenticated user, appended as the POST request parameter. If the submitting
user has an XMLRPC auth token with a description that matches this field, that
token is returned instead. The token is included in the POST data, and also in
an Authorization header.
* **dataset** This option specifies style of data that the system
will provide in the callback. It applies only for the POST request. The format
of the data and possible options are as following:
* **minimal** This will provide basic job info such as job id, status,
submit_time, start_time, end_time, submitter_username,
failure_comment, priority, description, actual_device_id, definition and
metadata.
* **logs** In addition to minimal data this will also attach the job log
output in the url encoded format
* **results** In addition to minimal data this will also attach the job
results as a list of test suites exported in yaml format.
* **all** In addition to minimal data this will include both the **logs** and
**results** datasets as described above.
* **content-type** This option is used to determine how the POST data is returned
* **urlencoded** (Default) Will return a standard HTTP POST request, with an
application/x-www-form-urlencoded Content-Type header and data sent as an
urlencoded query string.
* **json** The data is dumped into JSON and returned with an application/json
Content-Type header.
Example callback usage:
.. include:: examples/source/notifications.yaml
:code: yaml
:start-after: # notify callbacks block
Using profile settings
----------------------
LAVA users can configure the ``IRC settings`` or email address in their own
profile data in the instance. This allows the recipient to be specified using
only the LAVA username.
.. image:: images/profile-menu.png
Direct listing of recipients
-----------------------------
If the user has not configured their own profile data, the recipient details
must be specified in full.
Examples for user vs manual addressing:
.. include:: examples/source/notifications.yaml
:code: yaml
:start-after: # notify recipients block
:end-before: # notify compare block
IRC and email notifications use different templates since emails allow for more
verbosity, so some options which are present in notify block for email
recipients will be obsolete for IRC recipients, like **comparing** options and
**verbosity**.
Result comparison in notifications
==================================
Users can use notifications to compare the job status against the last few jobs
and also compare job results, like test case/suites difference. The
notification will report if there are new test cases compared to previous test
jobs, if there are some missing and how the said results differ.
Which test jobs the current one is compared to is determined by the
:ref:`queries` system, where those previous jobs will be taken from the query
specified in the notification block. Custom queries (:ref:`query_by_url`) can
also be specified.
**Blacklisting** test cases is also an option. Ignoring some of the test cases
in the comparing section of the notification is possible through an option in
the notification block.
Here are some comparing setup examples from test definition excerpts:
.. include:: examples/source/notifications.yaml
:code: yaml
:start-after: # notify compare block
:end-before: # notify compare custom block
.. include:: examples/source/notifications.yaml
:code: yaml
:start-after: # notify compare custom block
:end-before: # notify callbacks block
|
