aboutsummaryrefslogtreecommitdiff
path: root/automated/android
AgeCommit message (Collapse)Author
2020-09-05gearses2eclair.py: update to use regular expression to find the result lineYongqin Liu
to make the code a bit easy to read, and work with python3 Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2020-07-29Merge pull request #190 from liuyq/apk-automationmwasilew
apk-automation: update to work with aosp master branch
2020-07-29apk-automation: update to work with aosp master branchYongqin Liu
Tested locally with hikey960 aosp-master-throttled-copped#6640860 build Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2020-07-29boottime: add support aosp master versionYongqin Liu
collect ro.boottime properties and replace dc with awk tool and changed to get information from logcat dmesg buffer instead of dmesg log which might not have all logs Tested with following jobs: db845c: https://lkft.validation.linaro.org/scheduler/job/1598559 hikey960: https://validation.linaro.org/scheduler/job/2073798 x15: https://lkft.validation.linaro.org/scheduler/job/1596656 Problems: 1. launcher displayed log not printed for the second boot 2. it's not stable to reboot on lava, often failed to get the result for the second boot because of reboot problem. need to check from both lava and device side Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2020-07-04Merge pull request #179 from mwasilew/improve_self_testVishal Bhoj
Improve self test
2020-07-03tradefed: fix the problem for the removal of non xml supported charactersYongqin Liu
need to use the last position for next time when the matched pattern removed. otherwise, the next pattern just after the pattern matched this time might be ignored. Like this following example: test_helper.cc:55 &#27;[0m &#27;[31mConsumer unexpectedly disconnected from the service (errno: 0, Success)&#27;[0m after the first removal: test_helper.cc:55 [0m &#27;[31mConsumer unexpectedly disconnected from the service (errno: 0, Success)&#27;[0m we still need to search from the position after "test_helper.cc:55 ". Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2020-07-02tradefed.yaml: ignore error from ./tradefed.shYongqin Liu
so that the lines to upload result/log files could be executed, and we could get the result/logs files to check Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2020-06-30automated: manual: update format string to match LAVA expectationsMilosz Wasilewski
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2020-06-16ART benchmarks: switch to manifest prebuilt sdkJulien Duraj
2020-06-15ART: remove ART_REPORTS related code.jduraj
2020-03-12record boot to ui resultVishal Bhoj
Signed-off-by: Vishal Bhoj <vishal.bhoj@linaro.org>
2020-03-02fixed sanity-check warningsMilosz Wasilewski
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-12-02tradefed and noninteractive-tradefed: install different openjdk versionYongqin Liu
for different android version cts/vts test. Now only aosp master version needs to use openjdk-11, other older versions need to use openjdk8 Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2019-09-26automated/android: close popup when running GearsMilosz Wasilewski
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-09-26automated/android: fix apk-automation installation errorMilosz Wasilewski
New version of matplotlib requires python3. This patch makes sure that python2 version of matplotlib is installed Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-09-11uBenchmarks: provide target-device manuallyJulien Duraj
The script now autodetects the device, and could result in using binaries in a non-existing location.
2019-08-27android: optee: add hi6220-hikey-bl device to jobsVictor Chong
Change-Id: Ia11cca2bcca92fd33bce73d37d931659b897057d Signed-off-by: Victor Chong <victor.chong@linaro.org>
2019-08-27android: optee: add KMGK gtestVictor Chong
Change-Id: I3a29c90fc727da5d4f4ec7e89fe06391a8622eba Signed-off-by: Victor Chong <victor.chong@linaro.org>
2019-08-21android boottime: use wait_boot_completed instead of wait_homescreenYongqin Liu
as the launcher activity is not printed in the logcat any more Change-Id: Ia757ca5c9e9a7a63023e6ba6d55e3e6534d24b88 Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2019-07-26Added test definition to drive ion-monitor-tool for Android.Axel Le Bourhis
Also added an example test job for LAVA. This tool comes from MMWG team, to help monitoring memory leaks with ION heaps. Signed-off-by: Axel Le Bourhis <axel.lebourhis@nxp.com>
2019-07-23microbenchmarks: fix shellcheck errorMilosz Wasilewski
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-07-23microbenchmarks: add support for uploading to qa-reportsMilosz Wasilewski
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-07-11tradefed/noninteractive-tradefed: use jdk8 and remove _JAVA_OPTIONSYongqin Liu
1. the _JAVA_OPTIONS confuses the java version check in cts/vts and as cts/vts try to allocate more memory in their scripts following: https://android.googlesource.com/platform/test/vts/+/refs/heads/master/tools/vts-tradefed/etc/vts-tradefed#187 https://android.googlesource.com/platform/test/suite_harness/+/refs/heads/master/tools/cts-tradefed/etc/cts-tradefed#133 there is no meaning to set _JAVA_OPTIONS any more. 2. fixed to use jdk8 for vts/cts test as the installed openjdk-9-jdk-headless would cause following error with vts test: Using commandline arguments as starting command: [run, commandAndExit, vts-kernel, --module, VtsKernelLinuxKselftest] java.lang.reflect.InaccessibleObjectException: Unable to make member of class sun.reflect.annotation.AnnotationInvocationHandler accessible: module java.base does not export sun.reflect.annotation to unnamed module @3224f60b at sun.reflect.Reflection.throwInaccessibleObjectException(java.base@9-internal/Reflection.java:420) at java.lang.reflect.AccessibleObject.checkCanSetAccessible(java.base@9-internal/AccessibleObject.java:174) at java.lang.reflect.Field.checkCanSetAccessible(java.base@9-internal/Field.java:170) at java.lang.reflect.Field.setAccessible(java.base@9-internal/Field.java:164) at com.android.tradefed.config.OptionSetter.updateOptionChangedField(OptionSetter.java:556) at com.android.tradefed.config.OptionSetter.setFieldValue(OptionSetter.java:538) at com.android.tradefed.config.OptionSetter.setOptionValue(OptionSetter.java:414) at com.android.tradefed.config.OptionSetter.setOptionValue(OptionSetter.java:359) at com.android.compatibility.common.tradefed.testtype.suite.CompatibilityTestSuite.<init>(CompatibilityTestSuite.java:74) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(java.base@9-internal/Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(java.base@9-internal/NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(java.base@9-internal/DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(java.base@9-internal/Constructor.java:453) at java.lang.Class.newInstance(java.base@9-internal/Class.java:550) at com.android.tradefed.config.ConfigurationDef.createObject(ConfigurationDef.java:423) at com.android.tradefed.config.ConfigurationDef.createConfiguration(ConfigurationDef.java:252) at com.android.tradefed.config.ConfigurationFactory.internalCreateConfigurationFromArgs(ConfigurationFactory.java:573) at com.android.tradefed.config.ConfigurationFactory.createConfigurationFromArgs(ConfigurationFactory.java:495) at com.android.tradefed.command.CommandScheduler.createConfiguration(CommandScheduler.java:1191) at com.android.tradefed.command.CommandScheduler.internalAddCommand(CommandScheduler.java:1201) at com.android.tradefed.command.CommandScheduler.addCommand(CommandScheduler.java:1146) at com.android.tradefed.command.CommandScheduler.addCommand(CommandScheduler.java:1138) at com.android.tradefed.command.Console$27.run(Console.java:736) at com.android.tradefed.command.Console$27.run(Console.java:727) at com.android.tradefed.command.Console.executeCmdRunnable(Console.java:926) at com.android.tradefed.command.Console.run(Console.java:1028) at com.android.compatibility.common.tradefed.command.CompatibilityConsole.run(CompatibilityConsole.java:104) 07-10 04:46:57 V/LogRegistry: Saved log to /tmp/tradefed_global_log_6919477904182546528.txt after the change here: https://android-review.googlesource.com/c/platform/tools/tradefederation/+/1010290/3/src/com/android/tradefed/config/OptionSetter.java Change-Id: Ib972a8c3ca8c3e9b6ea2a07e03b42e357d487b34 Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2019-07-08microbenchmarks: fix path for post-build-report checkoutMilosz Wasilewski
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-07-05microbenchmarks: switch to official post-build-reportsMilosz Wasilewski
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-06-20tradefed monitor_fastboot.sh: support more boot image namesYongqin Liu
to work around problems like reported here: nice lxc-attach -n lxc-test-1919582 -- fastboot -s 28146262003EFFEE flash boot /lava-lxc/lkft-hikey-android-9.0-mainline-boot.img Thu Jun 20 08:45:49 UTC 2019 + fastboot boot /lava-lxc/boot*.img fastboot: error: cannot load '/lava-lxc/boot*.img': No such file or directory + sleep 30 https://validation.linaro.org/scheduler/job/1919582#L5627 Change-Id: Ie4e45e22ef203ea0c519979a2b411328ca3ccce9 Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2019-04-26boottime: run install_deps at the beginning of the scriptYongqin Liu
instead of in the middle of the test, especially not after any adb commands Change-Id: I3cadf14714ea9e69b5a1493597ac9be4e3bd444c Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2019-03-27ART: add few repos in manifestJulien Duraj
To use Google's ART buildbot scripts, we need to at least be able to source the AOSP environment. This results in a longer sync, but still reasonable (less than 45s).
2019-03-14MultiNode Tradefed: Reset userdata before retryKarsten Tausche
If supplied, flash a userdata image to the devices before triggering Tradefed retry. This helps reducing invalid failures that are caused by tests bringing devices into a state where other tests cannot pass. Issue: INFRA-137 Change-Id: I4a6510ceab6aec7ce530b9f8e244e9655dc22b81 Depends-On: Ie21cc417c78ec88da523f0d14ea85c1e9bcb912c Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-03-14MultiNode Tradefed: Fix udev access for fastbootKarsten Tausche
Execute the Tradefed runner as the current user, which is root in the context of LAVA LXC environments. This prevents getting udev permission issues in fastboot or adb, if udev rules are incomplete for specific Android devices and the device IDs they might report in different modes. Without this change, the `fastboot reboot` calls in the runner don't have any effect on affected configurations. Change-Id: Ie21cc417c78ec88da523f0d14ea85c1e9bcb912c Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-03-14Tradefed MultiNode: Refactor timeout handlingKarsten Tausche
Use timeout parameter of python subprocess commands instead of the system timeout command. Issue: INFRA-137 Change-Id: Id939f31d3d047a9d2c5d18ea4dd0151f04b567e4 Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-03-14Move wait-and-keep-local-device-accessible into shell scriptKarsten Tausche
Having the code in a dedicated script file rather than in YAML inline simplifies coding, testing and validation. This also fixes minor shellcheck and bashate issues. Issue: INFRA-137 Change-Id: Ia090f08eb325b3b348a4863cfde4a250f771a7e2 Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-03-14Allow reusing shell libs in Python 3 codeKarsten Tausche
This simplifies code reuse in the MultiNode Tradefed python runner and in upcoming changes. Issue: INFRA-135 Issue: INFRA-137 Change-Id: I6bbf2ca96cc74c2eabf7308c57be07064d81a8e4 Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-03-08android: tradefed-multinode: add lava job example for hikeyChase Qi
Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-03-08android: tradefed-multinode: fix test definitions repo linkChase Qi
Signed-off-by: Chase Qi <chase.qi@linaro.org>
2019-02-18Merge pull request #15 from kateyy/tradefed_shards_with_retryChase Qi
Add MultiNode TradeFed tests for --shards and with session retry
2019-02-18noninteractive-tradefed: use grep to workaround Bad substitution problemYongqin Liu
reported like following: /lava-610173/1/tests/1_cts-focused2-armeabi-v7a/run.sh: 35: /lava-610173/1/tests/1_cts-focused2-armeabi-v7a/run.sh: Bad substitution but not sure why this problem only happens for x15 jobs, jobs for hikey builds could work without problem Change-Id: I4dbaf0670f82bec69e2146d8cf91974feaf86dc1 Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
2019-02-15Add MultiNode TradeFed tests for --shards and with session retryKarsten Tausche
This builds upon the previous test setup and runner for TradeFed (CTS/VTS) with the following major features and changes: (1) Allow for "sharded" tests accross multiple DUTs (speedup test suite execution) (2) Use session retry for stabilizing final failure counts (3) Don't require DUT root access in any of the involved operations (4) Trying to be failure tolerant: Continue as long as at least one device is available. Re-establish lost connection if possible. (5) Drop support for VTS (temporary solution, it was not tested at all) (6) Add support for GTS and STS Details related to the points above: (1) Implemented using the LAVA MultiNode API. There should be one "master" job that executes the TradeFed runner and an arbitrary number of 'workers' that host additional DUTs and make them accessible via adb tcpip. Continuous messaging between master and worker is used to recover lost devices when needed. (3) Migrated features requiring root to non-root equivalents. (5) No testing was done with VTS, therefore it was partly dropped for now. (6) Apart from adjusted paths and command names, no changes were required compared to CTS. This change would ideally be a replacement of the existing TradeFed setup, or at least share more code with it. Remaining issues on that way are: - (5) - no VTS testing done. This relates to behavior of --retry, --shards, and related shell output - MultiNode environment expected by this setup. The runner could be further generalized to be usable from a regular (non-MultiNode) test submission, so that it would in a basic case not introduce any overhead compared to the existing runner. Change-Id: Idef4a5a9aac1f3cd8fc2aa1e609f544ee15ae528 Depends-On: I23f22344b9bd758d3898d4345204157cecd7d624 Depends-On: Ie011cb2cd899dac938066ca4eee7652b83ac38d4 Depends-On: I638d6a2cf44a5569569703308172e3056030783f Depends-On: I912ab5168bbe1b4fc0a3a8112db5cbc94a812b7c Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-02-15noninteractive-tradefed: add more debugMilosz Wasilewski
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-02-12Add MultiNode scripts for running tests on multiple adb TCP/IP devicesKarsten Tausche
With these test shells and functions, test implementations can make use of multiple devices by connecting remote devices via adb TCP/IP connections. On the MultiNode side, one master role would execute the actual test shell, whereas multiple instances of a worker role allow for remote access to their device adb TCP/IP. The following steps are required: * Start with completely booted and network attached devices. * workers: share-local-device-over-adb-tcpip.yaml - run `adb tcpip` on their local devices and share their IP address with the master * master: connect-to-remote-adb-tcpip-devices.yaml - Connects to all devices shared by the workers via `adb connect` (while also using its own local USB attached device) * workers: wait-and-keep-local-device-accessible.yaml - Wait for commands from the master -- mostly wait and do nothing else, but may need to reset their devices/network connections to make them accessible again if they got lost. * master: remote-adb-devices-smoke-test.yaml - Dummy action for the actual test job. A proper test job would execute some potentially long running tests on all available devices, instruct the worker to reconnect on connection lost etc. * master: release-remote-adb-tcpip-devices.yaml - Test executions end with the release command, so that workers exit their event loop. * workers: wait-for-release-and-reset.yaml - Final synchronization point between worker and the master, so that the master is in control when the workers are shutting down. Brings worker devices back into adb USB mode, so that they are usable for regular local test jobs. Change-Id: I23f22344b9bd758d3898d4345204157cecd7d624 Depends-On: Icd68e9de5a349880c52ec06229cd3f8bcb8eeecc Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2019-02-04android/tradefed: improve reboot variable comparisonMilosz Wasilewski
Make sure that comparison is case insensitive to catch any possible reboot in LAVA. Also align the comparison condition between different tradefed implementations. Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-02-01tradefed: fix comparison for case sensitive "True" valueMilosz Wasilewski
Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2019-01-08automated: android: optee: add 32-bit gtest for..Victor Chong
..gatekeeper and keymaster Change-Id: Ib998b465c85d2805542c0f0875536b1b69c150fd Signed-off-by: Victor Chong <victor.chong@linaro.org> Reviewed-by: Chase Qi <chase.qi@linaro.org>
2018-12-17Add utility yaml files to facilitate waiting for device availabilityKarsten Tausche
These yaml files simplify waiting for a specific device state in a test submission before running an actual test job. Change-Id: I0f6631afc1a73358e80220f6fc51789b0c4ec296 Depends-On: Icd68e9de5a349880c52ec06229cd3f8bcb8eeecc Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2018-12-14automated/android/tradefed: fix pep8 warningsMilosz Wasilewski
Change-Id: I94bd9dc5f50274a2a4ece1d9513fd896fce9bae0 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2018-12-14automated/android/tradefed: define class constantsMilosz Wasilewski
In order to avoid typing mistakes parser class defines two constant values that can be used with options and 'if' statements. Change-Id: I73eb6360b9f1d9bfa7c5b5eca8dc7fb94a3cfadc Signed-off-by: Milosz Wasilewski <milosz.wasilewski@linaro.org>
2018-12-14Merge "Extract result_parser.py from tradefed-runner.py"Milosz Wasilewski
2018-12-14automated: android: optee: add gtest for..Victor Chong
..gatekeeper and keymaster Change-Id: I24481cbfd27fc7a725abbf1e69840e28de1782f0 Signed-off-by: Victor Chong <victor.chong@linaro.org> Signed-off-by: Chase Qi <chase.qi@linaro.org>
2018-12-13Extract result_parser.py from tradefed-runner.pyKarsten Tausche
A first step to make the runner code more reusable by other implementations. Code from the previous result_parser() function and related constants were moved to result_parser.py to provide the parser code in a self-contained file. Additionally, the two printing approaches (aggregated, atomic) are now in dedicated functions to make the code more readable. Change-Id: I638d6a2cf44a5569569703308172e3056030783f Signed-off-by: Karsten Tausche <karsten@fairphone.com>
2018-12-12linaro-android-kernel-tests: disable selinux for testYongqin Liu
since this test is kernel related test, it might need premissions that not supported by default. It's not good to add rules for test purpose, which might affect other tests that, so we set it to permissve mode before we run this kernel test, and set it back to Enforce mode if it was before Change-Id: Ic02eb085bdbaeb37f0fd27339f1dd3939489aaab Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-09 06:42:42 +0000