diff options
| author | Pavel Begunkov <asml.silence@gmail.com> | 2023-01-05 12:22:28 +0100 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2023-01-29 23:17:40 +0100 |
| commit | 46ae7eef44f6dfd825a3bcfa43392d3ad9836ada (patch) | |
| tree | 8861a0baf334affd023c35b160f33d4ccbe66ba2 /io_uring | |
| parent | io_uring: set TASK_RUNNING right after schedule (diff) | |
| download | linux-46ae7eef44f6dfd825a3bcfa43392d3ad9836ada.tar.xz linux-46ae7eef44f6dfd825a3bcfa43392d3ad9836ada.zip | |
io_uring: optimise non-timeout waiting
Unlike the jiffy scheduling version, schedule_hrtimeout() jumps a few
functions before getting into schedule() even if there is no actual
timeout needed. Some tests showed that it takes up to 1% of CPU cycles.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/89f880574eceee6f4899783377ead234df7b3d04.1672916894.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'io_uring')
| -rw-r--r-- | io_uring/io_uring.c | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 2ec011f0ba7d..6229a49c0c33 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -2479,7 +2479,9 @@ static inline int io_cqring_wait_schedule(struct io_ring_ctx *ctx, return -EINTR; if (unlikely(io_should_wake(iowq))) return 0; - if (!schedule_hrtimeout(timeout, HRTIMER_MODE_ABS)) + if (*timeout == KTIME_MAX) + schedule(); + else if (!schedule_hrtimeout(timeout, HRTIMER_MODE_ABS)) return -ETIME; return 0; } |