diff options
author | Al Viro <viro@zeniv.linux.org.uk> | 2020-09-10 14:33:27 +0200 |
---|---|---|
committer | Al Viro <viro@zeniv.linux.org.uk> | 2020-09-10 23:14:44 +0200 |
commit | fe0a916c1eae8e17e86c3753d13919177d63ed7e (patch) | |
tree | 85f8db0d6e1081b98a953beaaca4c1d9afe305a8 /fs/eventpoll.c | |
parent | epoll: replace ->visited/visited_list with generation count (diff) | |
download | linux-fe0a916c1eae8e17e86c3753d13919177d63ed7e.tar.xz linux-fe0a916c1eae8e17e86c3753d13919177d63ed7e.zip |
epoll: EPOLL_CTL_ADD: close the race in decision to take fast path
Checking for the lack of epitems refering to the epoll we want to insert into
is not enough; we might have an insertion of that epoll into another one that
has already collected the set of files to recheck for excessive reverse paths,
but hasn't gotten to creating/inserting the epitem for it.
However, any such insertion in progress can be detected - it will update the
generation count in our epoll when it's done looking through it for files
to check. That gets done under ->mtx of our epoll and that allows us to
detect that safely.
We are *not* holding epmutex here, so the generation count is not stable.
However, since both the update of ep->gen by loop check and (later)
insertion into ->f_ep_link are done with ep->mtx held, we are fine -
the sequence is
grab epmutex
bump loop_check_gen
...
grab tep->mtx // 1
tep->gen = loop_check_gen
...
drop tep->mtx // 2
...
grab tep->mtx // 3
...
insert into ->f_ep_link
...
drop tep->mtx // 4
bump loop_check_gen
drop epmutex
and if the fastpath check in another thread happens for that
eventpoll, it can come
* before (1) - in that case fastpath is just fine
* after (4) - we'll see non-empty ->f_ep_link, slow path
taken
* between (2) and (3) - loop_check_gen is stable,
with ->mtx providing barriers and we end up taking slow path.
Note that ->f_ep_link emptiness check is slightly racy - we are protected
against insertions into that list, but removals can happen right under us.
Not a problem - in the worst case we'll end up taking a slow path for
no good reason.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Diffstat (limited to 'fs/eventpoll.c')
-rw-r--r-- | fs/eventpoll.c | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/fs/eventpoll.c b/fs/eventpoll.c index 82ab9a25f12f..16313180e4c1 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -2181,6 +2181,7 @@ int do_epoll_ctl(int epfd, int op, int fd, struct epoll_event *epds, goto error_tgt_fput; if (op == EPOLL_CTL_ADD) { if (!list_empty(&f.file->f_ep_links) || + ep->gen == loop_check_gen || is_file_epoll(tf.file)) { mutex_unlock(&ep->mtx); error = epoll_mutex_lock(&epmutex, 0, nonblock); |