aboutsummaryrefslogtreecommitdiff
Workflow:

0. Install all requirements and edit passwords
system dependencies for XML libs:
apt-get install python-dev libxml2-dev libxslt1-dev zlib1g-dev

virtualenv .venv
source .venv/bin/activate
pip install -r requirements.txt

Edit ~/.netrc to add passwords for
validation.linaro.org (LAVA token)
bugs.linaro.org (LDAP password)
android-qa-reports.linaro.org (LDAP password)

1. take a note of RC numbers
For exampel 16.09 Juno RC was: android-lcr-member-juno-n 45

2. get CTS job IDs:
./android-report-toolbox.py -b android-lcr-member-juno-n -n 45 -o m-lcr_16.09-juno cts-jobs

3. download CTS results
./cts-toolbox.py -o m-lcr_16.09-juno download -j <space separated list of LAVA IDs>

4. combine CTS results
./cts-toolbox.py -o m-lcr_16.09-juno combine

5. create CTS summary for amr64-v8a ABI
./cts-toolbox.py -o m-lcr_16.09-juno summary -a arm64-v8a

6. create CTS focused table for arm64-v8a ABI
./cts-toolbox.py -o m-lcr_16.09-juno focused -a arm64-v8a

7. create HTML report for CTS
./cts-toolbox.py -o m-lcr_16.09-juno html

8. download benchmark results
./android-report-toolbox.py -b android-lcr-member-juno-n -n 45 -o m-lcr_16.09-juno benchmarks

9. downlaod bugs
./android-report-toolbox.py -b android-lcr-member-juno-n -n 45 -o m-lcr_16.09-juno defects -d Juno

10. create index
./android-report-toolbox.py -b android-lcr-member-juno-n -n 45 -o m-lcr_16.09-juno index

After all of the above, there should be m-lcr_16.09-juno directory with the following contents:

ls -1 m-lcr_16.09-juno/
2016.09.21_02.45.47
2016.09.21_04.16.34
2016.09.21_08.49.23
2016.09.21_09.56.38
2016.09.21_10.14.00
2016.09.21_10.30.25
2016.09.21_11.17.10
2016.09.21_14.13.53
2016.09.21_17.36.41
benchmarks.rst
benchmarks.styles
bugs.rst
compatibility_result.css
cts-focused.rst
cts-focused.styles
cts-summary.rst
cts-summary.styles
index.rst
links.rst
logo.png
results.html
results.xml
test_result.xml

NOTE: Some additional manual editing is needed in order to prepare final LCR report. This work should be minimal.