diff options
author | Vincent Guittot <vincent.guittot@linaro.org> | 2021-11-09 16:25:22 +0100 |
---|---|---|
committer | Vincent Guittot <vincent.guittot@linaro.org> | 2022-01-20 14:36:24 +0100 |
commit | fcad8c03b07f1a62a7c4569511bc69ca573c80a9 (patch) | |
tree | 5e8e4d8f454fdfff2dce9572dc3ed7e24df368a2 /kernel/sched/sched.h | |
parent | 54e9ac6efb192eb31ecbc31cb1082b6dcc360cbc (diff) |
sched/fair: Take into account latency nice at wakeup
Take into account the nice latency priority of a thread when deciding to
preempt the current running thread. We don't want to provide more CPU
bandwidth to a thread but reorder the scheduling to run latency sensitive
task first whenever possible.
As long as a thread didn't use its bandwidth, it will be able to preempt
the current thread.
At the opposite, a thread with a low latency priority will preempt current
thread at wakeup only to keep fair CPU bandwidth sharing. Otherwise it will
wait for the tick to get its sched slice.
curr vruntime
|
sysctl_sched_wakeup_granularity
<-->
----------------------------------|----|-----------------------|---------------
| |<--------------------->
| . sysctl_sched_latency
| .
default/current latency entity | .
| .
1111111111111111111111111111111111|0000|-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-
se preempts curr at wakeup ------>|<- se doesn't preempt curr -----------------
| .
| .
| .
low latency entity | .
---------------------->|
% of sysctl_sched_latency |
1111111111111111111111111111111111111111111111111111111111|0000|-1-1-1-1-1-1-1-
preempt ------------------------------------------------->|<- do not preempt --
| .
| .
| .
high latency entity | .
|<-----------------------| .
| % of sysctl_sched_latency .
111111111|0000|-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1-1
preempt->|<- se doesn't preempt curr ------------------------------------------
Tests results of nice latency impact on heavy load like hackbench:
hackbench -l (2560 / group) -g group
group latency 0 latency 19
1 1.450(+/- 0.60%) 1.376(+/- 0.54%) + 5%
4 1.537(+/- 1.75%) 1.335(+/- 1.81%) +13%
8 1.414(+/- 1.91%) 1.348(+/- 1.88%) + 5%
16 1.423(+/- 1.65%) 1.374(+/- 0.58%) + 3%
hackbench -p -l (2560 / group) -g group
group
1 1.416(+/- 3.45%) 0.886(+/- 0.54%) +37%
4 1.634(+/- 7.14%) 0.888(+/- 5.40%) +45%
8 1.449(+/- 2.14%) 0.804(+/- 4.55%) +44%
16 0.917(+/- 4.12%) 0.777(+/- 1.41%) +15%
By deacreasing the latency prio, we reduce the number of preemption at
wakeup and help hackbench making progress.
Test results of nice latency impact on short live load like cyclictest
while competing with heavy load like hackbench:
hackbench -l 10000 -g group &
cyclictest --policy other -D 5 -q -n
latency 0 latency -20
group min avg max min avg max
0 16 17 28 15 17 27
1 61 382 10603 63 89 4628
4 52 437 15455 54 98 16238
8 56 728 38499 61 125 28983
16 53 1215 52207 61 183 80751
group = 0 means that hackbench is not running.
The avg is significantly improved with nice latency -20 especially with
large number of groups but min and max remain quite similar. If we add the
histogram parameters to get details of latency, we have :
hackbench -l 10000 -g 16 &
cyclictest --policy other -D 5 -q -n -H 20000 --histfile data.txt
latency 0 latency -20
Min Latencies: 63 62
Avg Latencies: 1397 218
Max Latencies: 44926 42291
50% latencies: 100 98
75% latencies: 762 116
85% latencies: 1118 126
90% latencies: 1540 130
95% latencies: 5610 138
99% latencies: 13738 266
With percentile details, we see the benefit of nice latency -20 as
1% of the latencies stays above 266us whereas the default latency has
got 25% are above 762us and 10% over the 1ms.
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Diffstat (limited to 'kernel/sched/sched.h')
-rw-r--r-- | kernel/sched/sched.h | 12 |
1 files changed, 12 insertions, 0 deletions
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index fa299862c319..21b683f3bd3d 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -122,6 +122,17 @@ extern void call_trace_sched_update_nr_running(struct rq *rq, int count); * Default tasks should be treated as a task with latency_nice = 0. */ #define DEFAULT_LATENCY_NICE 0 +#define DEFAULT_LATENCY_PRIO (DEFAULT_LATENCY_NICE + LATENCY_NICE_WIDTH/2) + +/* + * Convert user-nice values [ -20 ... 0 ... 19 ] + * to static latency [ 0..39 ], + * and back. + */ +#define NICE_TO_LATENCY(nice) ((nice) + DEFAULT_LATENCY_PRIO) +#define LATENCY_TO_NICE(prio) ((prio) - DEFAULT_LATENCY_PRIO) +#define NICE_LATENCY_SHIFT (SCHED_FIXEDPOINT_SHIFT) +#define NICE_LATENCY_WEIGHT_MAX (1L << NICE_LATENCY_SHIFT) /* * Increase resolution of nice-level calculations for 64-bit architectures. @@ -2098,6 +2109,7 @@ static_assert(WF_TTWU == SD_BALANCE_WAKE); extern const int sched_prio_to_weight[40]; extern const u32 sched_prio_to_wmult[40]; +extern const int sched_latency_to_weight[40]; /* * {de,en}queue flags: |