onsuccess: #17: 1: [TCWG CI] https://ci.linaro.org/job/tcwg_bmk-code_speed-cpu2017rate--gnu-aarch64-master-O2_LTO-build/17/

Results :
  | # reset_artifacts:
  | -10
  | # build_abe binutils:
  | -9
  | # build_abe stage1 -- --set gcc_override_configure=--disable-libsanitizer:
  | -8
  | # build_abe linux:
  | -7
  | # build_abe glibc:
  | -6
  | # build_abe stage2 -- --set gcc_override_configure=--disable-libsanitizer:
  | -5
  | # benchmark -- -O2_LTO:
  | 1

check_regression status : 0

3 files changed, 215 insertions, 179 deletions

diff --git a/09-benchmark/benchmark-build.log b/09-benchmark/benchmark-build.log
index 7703013..f38b46d 100644
--- a/09-benchmark/benchmark-build.log
+++ b/09-benchmark/benchmark-build.log
@@ -1,20 +1,20 @@
-Started TCWG Benchmark #26485
+Started TCWG Benchmark #26619
 Started from command line by tcwg-buildslave@linaro.org
 Running as SYSTEM
 [EnvInject] - Loading node environment variables.
-Building remotely on tcwg-bmk-bkp-01 (tcwg-bmk-coordinator tcwg-parallel-jobs tcwg) in workspace /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_8
+Building remotely on tcwg-bmk-bkp-01 (tcwg-bmk-coordinator tcwg-parallel-jobs tcwg) in workspace /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_3
 [ssh-agent] Looking for ssh-agent implementation...
 [ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
 $ ssh-agent
-SSH_AUTH_SOCK=/tmp/ssh-XXXXXXYHoPeq/agent.1386
-SSH_AGENT_PID=1388
+SSH_AUTH_SOCK=/tmp/ssh-XXXXXXNgY7XD/agent.57503
+SSH_AGENT_PID=57505
 [ssh-agent] Started.
 Running ssh-add (command line suppressed)
-Identity added: /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_8_tmp/private_key_11783341818787328013.key (/home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_8_tmp/private_key_11783341818787328013.key)
+Identity added: /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_3_tmp/private_key_13389927738264552334.key (/home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_3_tmp/private_key_13389927738264552334.key)
 [ssh-agent] Using credentials tcwg-benchmark (benchmark for TCWG machines)
 The recommended git tool is: NONE
 No credentials specified
- > /usr/bin/git rev-parse --resolve-git-dir /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_8/jenkins-scripts/.git # timeout=10
+ > /usr/bin/git rev-parse --resolve-git-dir /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_3/jenkins-scripts/.git # timeout=10
 Fetching changes from the remote Git repository
  > /usr/bin/git config remote.origin.url https://git.linaro.org/toolchain/jenkins-scripts.git # timeout=10
 Cleaning workspace
@@ -28,14 +28,14 @@ Fetching upstream changes from https://git.linaro.org/toolchain/jenkins-scripts.
  > git --version # 'git version 2.34.1'
  > /usr/bin/git fetch --tags --force --progress --prune -- https://git.linaro.org/toolchain/jenkins-scripts.git +refs/heads/*:refs/remotes/origin/* +refs/changes/*:refs/changes/* # timeout=10
  > /usr/bin/git rev-parse origin/master^{commit} # timeout=10
-Checking out Revision eacc707b4468dad30a5029b47fcb00ca080c03c5 (origin/master)
+Checking out Revision a943da76ab7ff09b6e8b7372aab8a4bd6d463611 (origin/master)
  > /usr/bin/git config core.sparsecheckout # timeout=10
- > /usr/bin/git checkout -f eacc707b4468dad30a5029b47fcb00ca080c03c5 # timeout=10
-Commit message: "tcwg_gnu-build.sh: Fix names of .exp file for tcwg_gnu_native_fast_check_gcc"
- > /usr/bin/git rev-list --no-walk eacc707b4468dad30a5029b47fcb00ca080c03c5 # timeout=10
+ > /usr/bin/git checkout -f a943da76ab7ff09b6e8b7372aab8a4bd6d463611 # timeout=10
+Commit message: "wrappers/install-wrappers.sh: Handle subdirectories under bin/"
+ > /usr/bin/git rev-list --no-walk a943da76ab7ff09b6e8b7372aab8a4bd6d463611 # timeout=10
 The recommended git tool is: NONE
 No credentials specified
- > /usr/bin/git rev-parse --resolve-git-dir /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_8/bmk-scripts/.git # timeout=10
+ > /usr/bin/git rev-parse --resolve-git-dir /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_3/bmk-scripts/.git # timeout=10
 Fetching changes from the remote Git repository
  > /usr/bin/git config remote.origin.url https://git.linaro.org/toolchain/bmk-scripts.git # timeout=10
 Cleaning workspace
@@ -49,95 +49,87 @@ Fetching upstream changes from https://git.linaro.org/toolchain/bmk-scripts.git
  > git --version # 'git version 2.34.1'
  > /usr/bin/git fetch --tags --force --progress --prune -- https://git.linaro.org/toolchain/bmk-scripts.git +refs/heads/*:refs/remotes/origin/* +refs/changes/*:refs/changes/* # timeout=10
  > /usr/bin/git rev-parse origin/master^{commit} # timeout=10
-Checking out Revision 385877571cab0862735fe6451025559f8b7f143f (origin/master)
+Checking out Revision ac7101f0f59415d62fd5edec58c751f7e81ad195 (origin/master)
  > /usr/bin/git config core.sparsecheckout # timeout=10
- > /usr/bin/git checkout -f 385877571cab0862735fe6451025559f8b7f143f # timeout=10
-Commit message: "output-bmk-results.py: Fix tabs and spaces in indentation"
- > /usr/bin/git rev-list --no-walk 385877571cab0862735fe6451025559f8b7f143f # timeout=10
+ > /usr/bin/git checkout -f ac7101f0f59415d62fd5edec58c751f7e81ad195 # timeout=10
+Commit message: "merge-metric-csvs.py: Add more checks to see if metric exists before using it."
+ > /usr/bin/git rev-list --no-walk ac7101f0f59415d62fd5edec58c751f7e81ad195 # timeout=10
 No emails were triggered.
-New run name is '#26485-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64)'
-[tcwg-benchmark-tcwg-bmk-bkp-01_8] $ /bin/bash /tmp/jenkins8429748025174843652.sh
+New run name is '#26619-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64)'
+[tcwg-benchmark-tcwg-bmk-bkp-01_3] $ /bin/bash /tmp/jenkins12673704327223293120.sh
 + rm -rf artifacts
 + mkdir -p artifacts/jenkins
 + has_perf_logs=yes
 + case "$bmk_hw:$image_arch" in
 + hw_tag=tx1_64
 ++ echo tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/@build_num@
-++ sed -e s/@build_num@/26485/g -e s/@hw_tag@/tx1_64/g
-+ results_id=tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
+++ sed -e s/@build_num@/26619/g -e s/@hw_tag@/tx1_64/g
++ results_id=tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + idle_boards=0
 + total_boards=0
 ++ set +f
 ++ ls -tr /home/tcwg-benchmark/boards/tcwg-bmk-tx1-01.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-02.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-03.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-04.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-05.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-06.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-08.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-09.lock
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1509
++ wait 57623
 + flock -en -E 124 9
-+ res=124
 + total_boards=1
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=1
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1510
++ wait 57624
 + flock -en -E 124 9
-+ res=124
 + total_boards=2
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=2
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1511
++ wait 57625
 + flock -en -E 124 9
-+ res=124
 + total_boards=3
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=3
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1512
++ wait 57626
 + flock -en -E 124 9
-+ res=124
 + total_boards=4
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=4
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1513
++ wait 57627
 + flock -en -E 124 9
-+ res=124
 + total_boards=5
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=5
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1514
++ wait 57628
 + flock -en -E 124 9
-+ res=124
 + total_boards=6
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=6
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1515
++ wait 57629
 + flock -en -E 124 9
-+ res=124
 + total_boards=7
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=7
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1516
++ wait 57630
 + flock -en -E 124 9
-+ res=124
 + total_boards=8
-+ '[' 124 = 124 ']'
-+ continue
-+ n_boards=4
-+ '[' 4 -lt 1 ']'
++ '[' 0 = 124 ']'
++ idle_boards=8
++ n_boards=8
++ '[' 8 -lt 1 ']'
 + count=1
 + read -a bmks
-++ ./bmk-scripts/break-up-bmks.sh --bench_list spec2017_rate --bmk_hw tx1 --n_boards 4 --run_profile serial --board_selection balance
+++ ./bmk-scripts/break-up-bmks.sh --bench_list spec2017_rate --bmk_hw tx1 --n_boards 8 --run_profile serial --board_selection balance
 + cat
 + count=2
 + read -a bmks
@@ -151,28 +143,52 @@ New run name is '#26485-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LT
 + count=5
 + read -a bmks
 + cat
++ count=6
++ read -a bmks
++ cat
++ count=7
++ read -a bmks
++ cat
++ count=8
++ read -a bmks
++ cat
++ count=9
++ read -a bmks
++ cat
+Waiting for the completion of TCWG Benchmark_backend
+TCWG Benchmark_backend #110095 started.
+TCWG Benchmark_backend #110095-26619+1-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+Waiting for the completion of TCWG Benchmark_backend
+TCWG Benchmark_backend #110096-26619+2-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110096-26619+2-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+Waiting for the completion of TCWG Benchmark_backend
+TCWG Benchmark_backend #110097-26619+3-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110097-26619+3-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+Waiting for the completion of TCWG Benchmark_backend
+TCWG Benchmark_backend #110098-26619+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110098-26619+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
 Waiting for the completion of TCWG Benchmark_backend
-TCWG Benchmark_backend #109681 started.
-TCWG Benchmark_backend #109681-26485+1-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+TCWG Benchmark_backend #110099-26619+5-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110099-26619+5-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
 Waiting for the completion of TCWG Benchmark_backend
-TCWG Benchmark_backend #109682-26485+2-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
-TCWG Benchmark_backend #109682-26485+2-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+TCWG Benchmark_backend #110100-26619+6-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110100-26619+6-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
 Waiting for the completion of TCWG Benchmark_backend
-TCWG Benchmark_backend #109683-26485+3-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
-TCWG Benchmark_backend #109683-26485+3-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+TCWG Benchmark_backend #110101-26619+7-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110101-26619+7-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
 Waiting for the completion of TCWG Benchmark_backend
-TCWG Benchmark_backend #109684-26485+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
-TCWG Benchmark_backend #109684-26485+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
-[tcwg-benchmark-tcwg-bmk-bkp-01_8] $ /bin/bash /tmp/jenkins2716810947361653786.sh
+TCWG Benchmark_backend #110102-26619+8-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110102-26619+8-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+[tcwg-benchmark-tcwg-bmk-bkp-01_3] $ /bin/bash /tmp/jenkins8120263274711677301.sh
 + . artifacts/bmk-info
-++ results_id=tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
+++ results_id=tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 ++ hw_tag=tx1_64
 ++ has_perf_logs=yes
-+ results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
-+ csv_results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
-+ rm -rf /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
-+ mkdir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
-+ ./bmk-scripts/gather-metrics.sh --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485 --csv_results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results --hw_tag tx1_64 --has_perf_logs yes
++ results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
++ csv_results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
++ rm -rf /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
++ mkdir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
++ ./bmk-scripts/gather-metrics.sh --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619 --csv_results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results --hw_tag tx1_64 --has_perf_logs yes
 ++ dirname ./bmk-scripts/gather-metrics.sh
 + bmk_scripts=./bmk-scripts
 + results_dir=undef
@@ -183,12 +199,12 @@ TCWG Benchmark_backend #109684-26485+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch
 + verbose=false
 + test 8 -gt 0
 + case $1 in
-+ results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + shift
 + shift
 + test 6 -gt 0
 + case $1 in
-+ csv_results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
++ csv_results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
 + shift
 + shift
 + test 4 -gt 0
@@ -202,14 +218,14 @@ TCWG Benchmark_backend #109684-26485+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch
 + shift
 + shift
 + test 0 -gt 0
-+ check_option /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485 'results_dir not set.'
-+ local var=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ check_option /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619 'results_dir not set.'
++ local var=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + local 'msg=results_dir not set.'
-+ '[' x/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485 = xundef ']'
-+ check_option /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results 'csv_results_dir not set.'
-+ local var=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
++ '[' x/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619 = xundef ']'
++ check_option /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results 'csv_results_dir not set.'
++ local var=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
 + local 'msg=csv_results_dir not set.'
-+ '[' x/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results = xundef ']'
++ '[' x/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results = xundef ']'
 + check_option tx1_64 'hw_tag not set.'
 + local var=tx1_64
 + local 'msg=hw_tag not set.'
@@ -220,43 +236,45 @@ TCWG Benchmark_backend #109684-26485+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch
 + '[' xyes = xundef ']'
 + '[' 0 = 0 ']'
 + metrics=(perf size vect sve md5sum)
-+ mkdir -p /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
++ mkdir -p /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
 + false
 + num_entries_opt=' '
-+ gather_perf_data yes tx1_64 /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485 /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
++ gather_perf_data yes tx1_64 /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619 /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
 + local has_perf_logs=yes
 + local hw_tag=tx1_64
-+ local results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
-+ local csv_results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
++ local results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
++ local csv_results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
 + '[' xyes = xyes ']'
-+ ./bmk-scripts/perfdatadir2csv.sh --buildid-dir local --format sample --sort-field sample --perf-bin /usr/lib/linux-tools/tx1_64/perf --results-dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ ./bmk-scripts/perfdatadir2csv.sh --buildid-dir local --format sample --sort-field sample --perf-bin /usr/lib/linux-tools/tx1_64/perf --results-dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + '[' xyes = xyes ']'
 + bmk_name=spec
 + metrics=("${metrics[@]:1}")
 + merge_opts=("$csv_results_dir/perf.csv")
 + for metric in "${metrics[@]}"
-+ ./bmk-scripts/size-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/size.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ ./bmk-scripts/size-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/size.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + merge_opts+=("$csv_results_dir/$metric.csv")
 + for metric in "${metrics[@]}"
-+ ./bmk-scripts/vect-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/vect.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ ./bmk-scripts/vect-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/vect.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + merge_opts+=("$csv_results_dir/$metric.csv")
 + for metric in "${metrics[@]}"
-+ ./bmk-scripts/sve-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/sve.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ ./bmk-scripts/sve-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/sve.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + merge_opts+=("$csv_results_dir/$metric.csv")
 + for metric in "${metrics[@]}"
-+ ./bmk-scripts/md5sum-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/md5sum.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ ./bmk-scripts/md5sum-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/md5sum.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + merge_opts+=("$csv_results_dir/$metric.csv")
-+ ./bmk-scripts/merge-metric-csvs.py /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/perf.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/size.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/vect.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/sve.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/md5sum.csv
-New run name is '#26485-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64)'
++ ./bmk-scripts/merge-metric-csvs.py /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/perf.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/size.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/vect.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/sve.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/md5sum.csv
+New run name is '#26619-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64)'
 $ ssh-agent -k
 unset SSH_AUTH_SOCK;
 unset SSH_AGENT_PID;
-echo Agent pid 1388 killed;
+echo Agent pid 57505 killed;
 [ssh-agent] Stopped.
 Archiving artifacts
 Email was triggered for: Success
 Sending email for trigger: Success
+#26618-tcwg_bmk-code_speed-cpu2017speed/llvm-aarch64-master-O3-build (apm:arm64) is still in progress; ignoring for purposes of comparison
 Failed to create e-mail address for ERROR:,File,'artifacts/jenkins/mail-recipients.txt',does,not,exist
+#26618-tcwg_bmk-code_speed-cpu2017speed/llvm-aarch64-master-O3-build (apm:arm64) is still in progress; ignoring for purposes of comparison
 An attempt to send an e-mail to empty list of recipients, ignored.
 Finished: SUCCESS
-Completed TCWG Benchmark #26485-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) : SUCCESS
+Completed TCWG Benchmark #26619-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) : SUCCESS
diff --git a/09-benchmark/benchmark.log b/09-benchmark/benchmark.log
index c27a523..2f4baf5 100644
--- a/09-benchmark/benchmark.log
+++ b/09-benchmark/benchmark.log
@@ -1,20 +1,20 @@
-Started TCWG Benchmark #26485
+Started TCWG Benchmark #26619
 Started from command line by tcwg-buildslave@linaro.org
 Running as SYSTEM
 [EnvInject] - Loading node environment variables.
-Building remotely on tcwg-bmk-bkp-01 (tcwg-bmk-coordinator tcwg-parallel-jobs tcwg) in workspace /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_8
+Building remotely on tcwg-bmk-bkp-01 (tcwg-bmk-coordinator tcwg-parallel-jobs tcwg) in workspace /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_3
 [ssh-agent] Looking for ssh-agent implementation...
 [ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
 $ ssh-agent
-SSH_AUTH_SOCK=/tmp/ssh-XXXXXXYHoPeq/agent.1386
-SSH_AGENT_PID=1388
+SSH_AUTH_SOCK=/tmp/ssh-XXXXXXNgY7XD/agent.57503
+SSH_AGENT_PID=57505
 [ssh-agent] Started.
 Running ssh-add (command line suppressed)
-Identity added: /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_8_tmp/private_key_11783341818787328013.key (/home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_8_tmp/private_key_11783341818787328013.key)
+Identity added: /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_3_tmp/private_key_13389927738264552334.key (/home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_3_tmp/private_key_13389927738264552334.key)
 [ssh-agent] Using credentials tcwg-benchmark (benchmark for TCWG machines)
 The recommended git tool is: NONE
 No credentials specified
- > /usr/bin/git rev-parse --resolve-git-dir /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_8/jenkins-scripts/.git # timeout=10
+ > /usr/bin/git rev-parse --resolve-git-dir /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_3/jenkins-scripts/.git # timeout=10
 Fetching changes from the remote Git repository
  > /usr/bin/git config remote.origin.url https://git.linaro.org/toolchain/jenkins-scripts.git # timeout=10
 Cleaning workspace
@@ -28,14 +28,14 @@ Fetching upstream changes from https://git.linaro.org/toolchain/jenkins-scripts.
  > git --version # 'git version 2.34.1'
  > /usr/bin/git fetch --tags --force --progress --prune -- https://git.linaro.org/toolchain/jenkins-scripts.git +refs/heads/*:refs/remotes/origin/* +refs/changes/*:refs/changes/* # timeout=10
  > /usr/bin/git rev-parse origin/master^{commit} # timeout=10
-Checking out Revision eacc707b4468dad30a5029b47fcb00ca080c03c5 (origin/master)
+Checking out Revision a943da76ab7ff09b6e8b7372aab8a4bd6d463611 (origin/master)
  > /usr/bin/git config core.sparsecheckout # timeout=10
- > /usr/bin/git checkout -f eacc707b4468dad30a5029b47fcb00ca080c03c5 # timeout=10
-Commit message: "tcwg_gnu-build.sh: Fix names of .exp file for tcwg_gnu_native_fast_check_gcc"
- > /usr/bin/git rev-list --no-walk eacc707b4468dad30a5029b47fcb00ca080c03c5 # timeout=10
+ > /usr/bin/git checkout -f a943da76ab7ff09b6e8b7372aab8a4bd6d463611 # timeout=10
+Commit message: "wrappers/install-wrappers.sh: Handle subdirectories under bin/"
+ > /usr/bin/git rev-list --no-walk a943da76ab7ff09b6e8b7372aab8a4bd6d463611 # timeout=10
 The recommended git tool is: NONE
 No credentials specified
- > /usr/bin/git rev-parse --resolve-git-dir /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_8/bmk-scripts/.git # timeout=10
+ > /usr/bin/git rev-parse --resolve-git-dir /home/tcwg-benchmark/workspace/tcwg-benchmark-tcwg-bmk-bkp-01_3/bmk-scripts/.git # timeout=10
 Fetching changes from the remote Git repository
  > /usr/bin/git config remote.origin.url https://git.linaro.org/toolchain/bmk-scripts.git # timeout=10
 Cleaning workspace
@@ -49,95 +49,87 @@ Fetching upstream changes from https://git.linaro.org/toolchain/bmk-scripts.git
  > git --version # 'git version 2.34.1'
  > /usr/bin/git fetch --tags --force --progress --prune -- https://git.linaro.org/toolchain/bmk-scripts.git +refs/heads/*:refs/remotes/origin/* +refs/changes/*:refs/changes/* # timeout=10
  > /usr/bin/git rev-parse origin/master^{commit} # timeout=10
-Checking out Revision 385877571cab0862735fe6451025559f8b7f143f (origin/master)
+Checking out Revision ac7101f0f59415d62fd5edec58c751f7e81ad195 (origin/master)
  > /usr/bin/git config core.sparsecheckout # timeout=10
- > /usr/bin/git checkout -f 385877571cab0862735fe6451025559f8b7f143f # timeout=10
-Commit message: "output-bmk-results.py: Fix tabs and spaces in indentation"
- > /usr/bin/git rev-list --no-walk 385877571cab0862735fe6451025559f8b7f143f # timeout=10
+ > /usr/bin/git checkout -f ac7101f0f59415d62fd5edec58c751f7e81ad195 # timeout=10
+Commit message: "merge-metric-csvs.py: Add more checks to see if metric exists before using it."
+ > /usr/bin/git rev-list --no-walk ac7101f0f59415d62fd5edec58c751f7e81ad195 # timeout=10
 No emails were triggered.
-New run name is '#26485-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64)'
-[tcwg-benchmark-tcwg-bmk-bkp-01_8] $ /bin/bash /tmp/jenkins8429748025174843652.sh
+New run name is '#26619-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64)'
+[tcwg-benchmark-tcwg-bmk-bkp-01_3] $ /bin/bash /tmp/jenkins12673704327223293120.sh
 + rm -rf artifacts
 + mkdir -p artifacts/jenkins
 + has_perf_logs=yes
 + case "$bmk_hw:$image_arch" in
 + hw_tag=tx1_64
 ++ echo tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/@build_num@
-++ sed -e s/@build_num@/26485/g -e s/@hw_tag@/tx1_64/g
-+ results_id=tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
+++ sed -e s/@build_num@/26619/g -e s/@hw_tag@/tx1_64/g
++ results_id=tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + idle_boards=0
 + total_boards=0
 ++ set +f
 ++ ls -tr /home/tcwg-benchmark/boards/tcwg-bmk-tx1-01.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-02.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-03.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-04.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-05.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-06.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-08.lock /home/tcwg-benchmark/boards/tcwg-bmk-tx1-09.lock
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1509
++ wait 57623
 + flock -en -E 124 9
-+ res=124
 + total_boards=1
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=1
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1510
++ wait 57624
 + flock -en -E 124 9
-+ res=124
 + total_boards=2
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=2
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1511
++ wait 57625
 + flock -en -E 124 9
-+ res=124
 + total_boards=3
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=3
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1512
++ wait 57626
 + flock -en -E 124 9
-+ res=124
 + total_boards=4
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=4
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1513
++ wait 57627
 + flock -en -E 124 9
-+ res=124
 + total_boards=5
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=5
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1514
++ wait 57628
 + flock -en -E 124 9
-+ res=124
 + total_boards=6
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=6
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1515
++ wait 57629
 + flock -en -E 124 9
-+ res=124
 + total_boards=7
-+ '[' 124 = 124 ']'
-+ continue
++ '[' 0 = 124 ']'
++ idle_boards=7
 + for lock in $(set +f; ls -tr $HOME/boards/tcwg-bmk-$bmk_hw-*.lock)
 + res=0
-+ wait 1516
++ wait 57630
 + flock -en -E 124 9
-+ res=124
 + total_boards=8
-+ '[' 124 = 124 ']'
-+ continue
-+ n_boards=4
-+ '[' 4 -lt 1 ']'
++ '[' 0 = 124 ']'
++ idle_boards=8
++ n_boards=8
++ '[' 8 -lt 1 ']'
 + count=1
 + read -a bmks
-++ ./bmk-scripts/break-up-bmks.sh --bench_list spec2017_rate --bmk_hw tx1 --n_boards 4 --run_profile serial --board_selection balance
+++ ./bmk-scripts/break-up-bmks.sh --bench_list spec2017_rate --bmk_hw tx1 --n_boards 8 --run_profile serial --board_selection balance
 + cat
 + count=2
 + read -a bmks
@@ -151,28 +143,52 @@ New run name is '#26485-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LT
 + count=5
 + read -a bmks
 + cat
++ count=6
++ read -a bmks
++ cat
++ count=7
++ read -a bmks
++ cat
++ count=8
++ read -a bmks
++ cat
++ count=9
++ read -a bmks
++ cat
+Waiting for the completion of TCWG Benchmark_backend
+TCWG Benchmark_backend #110095 started.
+TCWG Benchmark_backend #110095-26619+1-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+Waiting for the completion of TCWG Benchmark_backend
+TCWG Benchmark_backend #110096-26619+2-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110096-26619+2-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+Waiting for the completion of TCWG Benchmark_backend
+TCWG Benchmark_backend #110097-26619+3-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110097-26619+3-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+Waiting for the completion of TCWG Benchmark_backend
+TCWG Benchmark_backend #110098-26619+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110098-26619+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
 Waiting for the completion of TCWG Benchmark_backend
-TCWG Benchmark_backend #109681 started.
-TCWG Benchmark_backend #109681-26485+1-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+TCWG Benchmark_backend #110099-26619+5-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110099-26619+5-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
 Waiting for the completion of TCWG Benchmark_backend
-TCWG Benchmark_backend #109682-26485+2-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
-TCWG Benchmark_backend #109682-26485+2-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+TCWG Benchmark_backend #110100-26619+6-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110100-26619+6-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
 Waiting for the completion of TCWG Benchmark_backend
-TCWG Benchmark_backend #109683-26485+3-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
-TCWG Benchmark_backend #109683-26485+3-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+TCWG Benchmark_backend #110101-26619+7-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110101-26619+7-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
 Waiting for the completion of TCWG Benchmark_backend
-TCWG Benchmark_backend #109684-26485+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
-TCWG Benchmark_backend #109684-26485+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
-[tcwg-benchmark-tcwg-bmk-bkp-01_8] $ /bin/bash /tmp/jenkins2716810947361653786.sh
+TCWG Benchmark_backend #110102-26619+8-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) started.
+TCWG Benchmark_backend #110102-26619+8-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64) completed. Result was SUCCESS
+[tcwg-benchmark-tcwg-bmk-bkp-01_3] $ /bin/bash /tmp/jenkins8120263274711677301.sh
 + . artifacts/bmk-info
-++ results_id=tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
+++ results_id=tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 ++ hw_tag=tx1_64
 ++ has_perf_logs=yes
-+ results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
-+ csv_results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
-+ rm -rf /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
-+ mkdir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
-+ ./bmk-scripts/gather-metrics.sh --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485 --csv_results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results --hw_tag tx1_64 --has_perf_logs yes
++ results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
++ csv_results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
++ rm -rf /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
++ mkdir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
++ ./bmk-scripts/gather-metrics.sh --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619 --csv_results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results --hw_tag tx1_64 --has_perf_logs yes
 ++ dirname ./bmk-scripts/gather-metrics.sh
 + bmk_scripts=./bmk-scripts
 + results_dir=undef
@@ -183,12 +199,12 @@ TCWG Benchmark_backend #109684-26485+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch
 + verbose=false
 + test 8 -gt 0
 + case $1 in
-+ results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + shift
 + shift
 + test 6 -gt 0
 + case $1 in
-+ csv_results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
++ csv_results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
 + shift
 + shift
 + test 4 -gt 0
@@ -202,14 +218,14 @@ TCWG Benchmark_backend #109684-26485+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch
 + shift
 + shift
 + test 0 -gt 0
-+ check_option /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485 'results_dir not set.'
-+ local var=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ check_option /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619 'results_dir not set.'
++ local var=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + local 'msg=results_dir not set.'
-+ '[' x/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485 = xundef ']'
-+ check_option /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results 'csv_results_dir not set.'
-+ local var=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
++ '[' x/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619 = xundef ']'
++ check_option /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results 'csv_results_dir not set.'
++ local var=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
 + local 'msg=csv_results_dir not set.'
-+ '[' x/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results = xundef ']'
++ '[' x/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results = xundef ']'
 + check_option tx1_64 'hw_tag not set.'
 + local var=tx1_64
 + local 'msg=hw_tag not set.'
@@ -220,42 +236,44 @@ TCWG Benchmark_backend #109684-26485+4-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch
 + '[' xyes = xundef ']'
 + '[' 0 = 0 ']'
 + metrics=(perf size vect sve md5sum)
-+ mkdir -p /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
++ mkdir -p /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
 + false
 + num_entries_opt=' '
-+ gather_perf_data yes tx1_64 /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485 /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
++ gather_perf_data yes tx1_64 /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619 /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
 + local has_perf_logs=yes
 + local hw_tag=tx1_64
-+ local results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
-+ local csv_results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results
++ local results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
++ local csv_results_dir=/home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results
 + '[' xyes = xyes ']'
-+ ./bmk-scripts/perfdatadir2csv.sh --buildid-dir local --format sample --sort-field sample --perf-bin /usr/lib/linux-tools/tx1_64/perf --results-dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ ./bmk-scripts/perfdatadir2csv.sh --buildid-dir local --format sample --sort-field sample --perf-bin /usr/lib/linux-tools/tx1_64/perf --results-dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + '[' xyes = xyes ']'
 + bmk_name=spec
 + metrics=("${metrics[@]:1}")
 + merge_opts=("$csv_results_dir/perf.csv")
 + for metric in "${metrics[@]}"
-+ ./bmk-scripts/size-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/size.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ ./bmk-scripts/size-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/size.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + merge_opts+=("$csv_results_dir/$metric.csv")
 + for metric in "${metrics[@]}"
-+ ./bmk-scripts/vect-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/vect.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ ./bmk-scripts/vect-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/vect.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + merge_opts+=("$csv_results_dir/$metric.csv")
 + for metric in "${metrics[@]}"
-+ ./bmk-scripts/sve-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/sve.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ ./bmk-scripts/sve-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/sve.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + merge_opts+=("$csv_results_dir/$metric.csv")
 + for metric in "${metrics[@]}"
-+ ./bmk-scripts/md5sum-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/md5sum.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485
++ ./bmk-scripts/md5sum-data-to-csv.py --bmk_name spec --out_csv_file /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/md5sum.csv --perf_csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/perf.csv --results_dir /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619
 + merge_opts+=("$csv_results_dir/$metric.csv")
-+ ./bmk-scripts/merge-metric-csvs.py /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/perf.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/size.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/vect.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/sve.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26485/csv_results/md5sum.csv
-New run name is '#26485-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64)'
++ ./bmk-scripts/merge-metric-csvs.py /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/perf.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/size.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/vect.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/sve.csv /home/tcwg-benchmark/results-tx1_64/tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build/26619/csv_results/md5sum.csv
+New run name is '#26619-tcwg_bmk-code_speed-cpu2017rate/gnu-aarch64-master-O2_LTO-build (tx1:arm64)'
 $ ssh-agent -k
 unset SSH_AUTH_SOCK;
 unset SSH_AGENT_PID;
-echo Agent pid 1388 killed;
+echo Agent pid 57505 killed;
 [ssh-agent] Stopped.
 Archiving artifacts
 Email was triggered for: Success
 Sending email for trigger: Success
+#26618-tcwg_bmk-code_speed-cpu2017speed/llvm-aarch64-master-O3-build (apm:arm64) is still in progress; ignoring for purposes of comparison
 Failed to create e-mail address for ERROR:,File,'artifacts/jenkins/mail-recipients.txt',does,not,exist
+#26618-tcwg_bmk-code_speed-cpu2017speed/llvm-aarch64-master-O3-build (apm:arm64) is still in progress; ignoring for purposes of comparison
 An attempt to send an e-mail to empty list of recipients, ignored.
 Finished: SUCCESS
diff --git a/09-benchmark/console.log.xz b/09-benchmark/console.log.xz
index 3b15189..4a6b470 100644
--- a/09-benchmark/console.log.xz
+++ b/09-benchmark/console.log.xz