diff options
| author | Peter Zijlstra <peterz@infradead.org> | 2018-08-26 12:56:48 +0200 |
|---|---|---|
| committer | Peter Zijlstra <peterz@infradead.org> | 2018-10-09 16:51:11 +0200 |
| commit | a31acd3ee8f7dbc0370bdf4a4bfef7a8c13c7542 (patch) | |
| tree | fe6035ec072385441ded002ba945960ba1ddc034 /arch/x86/include/asm/tlb.h | |
| parent | Merge branch 'tlb/asm-generic' of git://git.kernel.org/pub/scm/linux/kernel/g... (diff) | |
| download | linux-a31acd3ee8f7dbc0370bdf4a4bfef7a8c13c7542.tar.xz linux-a31acd3ee8f7dbc0370bdf4a4bfef7a8c13c7542.zip | |
x86/mm: Page size aware flush_tlb_mm_range()
Use the new tlb_get_unmap_shift() to determine the stride of the
INVLPG loop.
Cc: Nick Piggin <npiggin@gmail.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Diffstat (limited to 'arch/x86/include/asm/tlb.h')
| -rw-r--r-- | arch/x86/include/asm/tlb.h | 21 |
1 files changed, 14 insertions, 7 deletions
diff --git a/arch/x86/include/asm/tlb.h b/arch/x86/include/asm/tlb.h index cb0a1f470980..afbe7d1e68cf 100644 --- a/arch/x86/include/asm/tlb.h +++ b/arch/x86/include/asm/tlb.h @@ -6,16 +6,23 @@ #define tlb_end_vma(tlb, vma) do { } while (0) #define __tlb_remove_tlb_entry(tlb, ptep, address) do { } while (0) -#define tlb_flush(tlb) \ -{ \ - if (!tlb->fullmm && !tlb->need_flush_all) \ - flush_tlb_mm_range(tlb->mm, tlb->start, tlb->end, 0UL); \ - else \ - flush_tlb_mm_range(tlb->mm, 0UL, TLB_FLUSH_ALL, 0UL); \ -} +static inline void tlb_flush(struct mmu_gather *tlb); #include <asm-generic/tlb.h> +static inline void tlb_flush(struct mmu_gather *tlb) +{ + unsigned long start = 0UL, end = TLB_FLUSH_ALL; + unsigned int stride_shift = tlb_get_unmap_shift(tlb); + + if (!tlb->fullmm && !tlb->need_flush_all) { + start = tlb->start; + end = tlb->end; + } + + flush_tlb_mm_range(tlb->mm, start, end, stride_shift); +} + /* * While x86 architecture in general requires an IPI to perform TLB * shootdown, enablement code for several hypervisors overrides |