diff options
| author | Heiko Carstens <hca@linux.ibm.com> | 2023-03-14 14:28:08 +0100 |
|---|---|---|
| committer | Andrew Morton <akpm@linux-foundation.org> | 2023-04-06 05:03:02 +0200 |
| commit | e06f47a16573decc57498f2d02f9af3bb3e84cf2 (patch) | |
| tree | 51722657e1d38694ced1774c2fb4ed108bb72834 /arch/s390/mm | |
| parent | mm: separate vma->lock from vm_area_struct (diff) | |
| download | linux-e06f47a16573decc57498f2d02f9af3bb3e84cf2.tar.xz linux-e06f47a16573decc57498f2d02f9af3bb3e84cf2.zip | |
s390/mm: try VMA lock-based page fault handling first
Attempt VMA lock-based page fault handling first, and fall back to the
existing mmap_lock-based handling if that fails.
This is the s390 variant of "x86/mm: try VMA lock-based page fault handling
first".
Link: https://lkml.kernel.org/r/20230314132808.1266335-1-hca@linux.ibm.com
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Diffstat (limited to 'arch/s390/mm')
| -rw-r--r-- | arch/s390/mm/fault.c | 24 |
1 files changed, 24 insertions, 0 deletions
diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c index a2632fd97d00..b65144c392b0 100644 --- a/arch/s390/mm/fault.c +++ b/arch/s390/mm/fault.c @@ -407,6 +407,30 @@ static inline vm_fault_t do_exception(struct pt_regs *regs, int access) access = VM_WRITE; if (access == VM_WRITE) flags |= FAULT_FLAG_WRITE; +#ifdef CONFIG_PER_VMA_LOCK + if (!(flags & FAULT_FLAG_USER)) + goto lock_mmap; + vma = lock_vma_under_rcu(mm, address); + if (!vma) + goto lock_mmap; + if (!(vma->vm_flags & access)) { + vma_end_read(vma); + goto lock_mmap; + } + fault = handle_mm_fault(vma, address, flags | FAULT_FLAG_VMA_LOCK, regs); + vma_end_read(vma); + if (!(fault & VM_FAULT_RETRY)) { + count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + goto out; + } + count_vm_vma_lock_event(VMA_LOCK_RETRY); + /* Quick path to respond to signals */ + if (fault_signal_pending(fault, regs)) { + fault = VM_FAULT_SIGNAL; + goto out; + } +lock_mmap: +#endif /* CONFIG_PER_VMA_LOCK */ mmap_read_lock(mm); gmap = NULL; |