diff options
| author | Vincent Guittot <vincent.guittot@linaro.org> | 2021-06-01 10:58:32 +0200 |
|---|---|---|
| committer | Peter Zijlstra <peterz@infradead.org> | 2021-06-03 12:55:55 +0200 |
| commit | fcf6631f3736985ec89bdd76392d3c7bfb60119f (patch) | |
| tree | 22924383c5f369cfcd524254379cf6f86a7b6dc1 /kernel/sched | |
| parent | tick/nohz: Only check for RCU deferred wakeup on user/guest entry when needed (diff) | |
| download | linux-fcf6631f3736985ec89bdd76392d3c7bfb60119f.tar.xz linux-fcf6631f3736985ec89bdd76392d3c7bfb60119f.zip | |
sched/pelt: Ensure that *_sum is always synced with *_avg
Rounding in PELT calculation happening when entities are attached/detached
of a cfs_rq can result into situations where util/runnable_avg is not null
but util/runnable_sum is. This is normally not possible so we need to
ensure that util/runnable_sum stays synced with util/runnable_avg.
detach_entity_load_avg() is the last place where we don't sync
util/runnable_sum with util/runnbale_avg when moving some sched_entities
Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20210601085832.12626-1-vincent.guittot@linaro.org
Diffstat (limited to 'kernel/sched')
| -rw-r--r-- | kernel/sched/fair.c | 10 |
1 files changed, 8 insertions, 2 deletions
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e7c8277e3d54..7b98fb37330a 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -3765,11 +3765,17 @@ static void attach_entity_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *s */ static void detach_entity_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se) { + /* + * cfs_rq->avg.period_contrib can be used for both cfs_rq and se. + * See ___update_load_avg() for details. + */ + u32 divider = get_pelt_divider(&cfs_rq->avg); + dequeue_load_avg(cfs_rq, se); sub_positive(&cfs_rq->avg.util_avg, se->avg.util_avg); - sub_positive(&cfs_rq->avg.util_sum, se->avg.util_sum); + cfs_rq->avg.util_sum = cfs_rq->avg.util_avg * divider; sub_positive(&cfs_rq->avg.runnable_avg, se->avg.runnable_avg); - sub_positive(&cfs_rq->avg.runnable_sum, se->avg.runnable_sum); + cfs_rq->avg.runnable_sum = cfs_rq->avg.runnable_avg * divider; add_tg_cfs_propagate(cfs_rq, -se->avg.load_sum); |