1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
|
h1{color:blue;}. Test Plan for Linaro OpenEmbedded RPB release for Qualcomm(tm) Dragonboard(tm) 410c
!{width: 20%; padding: 5px; }https://www.linaro.org/assets/images/Linaro-Logo.svg(Linaro Logo)!
*Date:* August 16, 2021
h1. Executive Summary
This document describes the testing Linaro will carry out when making a release of the Linaro OpenEmbedded RPB release for Qualcomm(tm) Dragonboard(tm) 410c.
h1. Hardware Platforms
Linaro provide releases for the following platforms from Qualcomm:
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Board |
| Dragonboard410c |
h1. Tests definitions
Linaro uses LAVA test definition syntaxt to create and manage automated and manual tests. To learn more about LAVA test definition, you can refer to LAVA Documentation, in particular: "Writing a Lava-Test Test Definition 1.0¶
":https://docs.lavasoftware.org/lava/writing-tests.html.
Linaro test cases are archived in the upstream Linaro Test Definitions repository here: "https://github.com/Linaro/test-definitions/":https://github.com/Linaro/test-definitions.
h1. Automated Tests
Linaro runs automated testing in the LAVA lab.
This section describes the automated tests used from the Linaro Test Definitions repository.
h2. 1. linux-smoke
Run basic smoke tests.
_Test definition:_ "automated/linux/smoke/smoke.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/smoke/smoke.yaml
_Test scope:_ functional
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| SKIP_INSTALL | False |
| TESTS | pwd, lsb_release -a, uname -a, ip a, lscpu, vmstat, lsblk |
h2. 2. linux-usb-smoke
Basic USB test for Linaro Ubuntu images. The test examines all available USB devices and prints supported USB protocols and speed.
_Test definition:_ "automated/linux/usb-smoke/usb-smoke-test.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/usb-smoke/usb-smoke-test.yaml
_Test scope:_ functional
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| SKIP_INSTALL | false |
h2. 3. linux-toolchain-smoke
Simple test that checks whether gcc compilation works
_Test definition:_ "automated/linux/toolchain-smoke/toolchain-smoke.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/toolchain-smoke/toolchain-smoke.yaml
_Test scope:_ functional
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| STATIC | False |
| SKIP_INSTALL | False |
h2. 4. linux-device-tree
Device tree test to check the folder structure. The test verifies that device-tree is available and contains model name of the board.
_Test definition:_ "automated/linux/device-tree/device-tree.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/device-tree/device-tree.yaml
_Test scope:_ functional
h2. 5. meminfo
Collect the information in /proc/meminfo after boot.
_Test definition:_ "automated/linux/meminfo/meminfo.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/meminfo/meminfo.yaml
_Test scope:_ functional
h2. 6. linux-dd-wr-speed
dd write/read speed test.
_Test definition:_ "automated/linux/dd-wr-speed/dd-wr-speed.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/dd-wr-speed/dd-wr-speed.yaml
_Test scope:_ performance
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| PARTITION | |
| FS_TYPE | |
| ITERATION | 5 |
| SKIP_INSTALL | false |
h2. 7. linux-sysbench
SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load. Current features allow to test fileio, cpu, memory, threads, mutex and oltp.
_Test definition:_ "automated/linux/sysbench/sysbench.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/sysbench/sysbench.yaml
_Test scope:_ performance
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| TESTS | percpu cpu memory threads mutex fileio |
| NUM_THREADS | NPROC |
| SKIP_INSTALL | false |
h2. 8. linux-device-read-perf
Use 'hdparm -t' to perform timings of block device reads for benchmark and comparison purposes.
_Test definition:_ "automated/linux/device-read-perf/device-read-perf.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/device-read-perf/device-read-perf.yaml
_Test scope:_ performance
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| DEVICE | |
| SKIP_INSTALL | False |
h2. 9. linux-hackbench
Hackbench is both a benchmark and a stress test for the Linux kernel scheduler. It's main job is to create a specified number of pairs of schedulable entities (either threads or traditional processes) which communicate via either sockets or pipes and time how long it takes for each pair to send data back and forth.
_Test definition:_ "automated/linux/hackbench/hackbench.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/hackbench/hackbench.yaml
_Test scope:_ performance
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| ITERATION | 1000 |
| TARGET | host |
| DATASIZE | 100 |
| LOOPS | 100 |
| GRPS | 10 |
| FDS | 20 |
| PIPE | false |
| THREADS | false |
h2. 10. linux-linpack
Linpack benchmark calculates fLoating point operations per second(FLOPS).
_Test definition:_ "automated/linux/linpack/linpack.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/linpack/linpack.yaml
_Test scope:_ performance
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| ARRAY_SIZE | 200 |
h2. 11. linux-openssl-speed
Use openssl speed command to test the performance of cryptographic algorithms
_Test definition:_ "automated/linux/openssl/openssl-speed.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/openssl/openssl-speed.yaml
_Test scope:_ performance
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| SKIP_INSTALL | False |
h2. 12. linux-stream-uniprocessor
STREAM benchmark is a simple synthetic benchmark program that measures sustainable memory bandwidth (in MB/s). This test case run STREAM on a single processor.
_Test definition:_ "automated/linux/stream/stream-uniprocessor.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/stream/stream-uniprocessor.yaml
_Test scope:_ performance
h2. 13. linux-pi-stress
pi_stress checks Priority Inheritence Mutexes and their ability to avoid Priority Inversion from occuring by running groups of threads that cause Priority Inversions.
_Test definition:_ "automated/linux/pi-stress/pi-stress.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/pi-stress/pi-stress.yaml
_Test scope:_ functional
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| DURATION | 5m |
| MLOCKALL | false |
| RR | false |
| BACKGROUND_CMD | |
h2. 14. linux-pm-qa
Test power management (PM-QA). Currently, the test runs cpufreq, cpuidle, cpuhotplug, thermal and cputopology by default and individual test can be run by setting TESTS parameter in test job definition too. Test case documentation available in wiki: https://wiki.linaro.org/WorkingGroups/PowerManagement/Resources/TestSuite/PmQaSpecification
_Test definition:_ "automated/linux/pm-qa/pm-qa.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/pm-qa/pm-qa.yaml
_Test scope:_ functional
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| RELEASE | pm-qa-v0.5.2 |
| TESTS | cpufreq cpuidle cpuhotplug thermal cputopology |
| SKIP_INSTALL | false |
h2. 15. linux-containers
Docker smoke test. When docker not pre-installed, docker-ce will be installed from upstream on supported distributions.
_Test definition:_ "automated/linux/docker/docker.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/docker/docker.yaml
_Test scope:_ functional
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| IMAGE | aarch64/ubuntu |
h2. 16. linux-wlan-smoke
Basic WLAN smoke tests
_Test definition:_ "automated/linux/wlan-smoke/wlan-smoke.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/wlan-smoke/wlan-smoke.yaml
_Test scope:_ functional
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| DEVICE | wlan0 |
| BOOT | enabled |
h2. 17. wlan-download
Basic WLAN download test, requires ip, iw, wpa_supplicant and dhclient
_Test definition:_ "automated/linux/wlan-download/wlan-download.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/wlan-download/wlan-download.yaml
_Test scope:_ functional
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| DEVICE | wlan0 |
| ETHERNET_DEVICE | |
| SSID_NAME | |
| SSID_PASSWORD | |
| FILE_URL | |
| FILE_CHECKSUM | |
| TIME_DELAY | 0s |
| NAMESERVER | |
h2. 18. linux-hci-smoke
Basic HCI smoke tests The test will run the following commands: - hciconfig up/down - hcitool scan - hcitool lescan (when le mode is enabled) - hcitool inq When testing LE mode some helper BT device must be in DUT's proximity.
_Test definition:_ "automated/linux/hci-smoke/hci-smoke.yaml":https://git.linaro.org/qa/test-definitions.git/tree/66202972f3dde68614689e43bd7a094e8e314176/automated/linux/hci-smoke/hci-smoke.yaml
_Test scope:_ functional
table{width: 50%; border:1px solid black; border-spacing: 10px; text-align: left; border-collapse:collapse}.
{background:#ddd}. |_. Parameter |_. Value |
| DEVICE | hci0 |
| BOOT | enabled |
| MODE | scan |
|
