diff options
author | Steve Capper <steve.capper@linaro.org> | 2013-10-07 10:47:28 +0100 |
---|---|---|
committer | Kim Phillips <kim.phillips@linaro.org> | 2013-10-10 10:57:08 -0500 |
commit | 2c0f5f40b4d3dd5d2ba0c49356c97d8bc72ab30f (patch) | |
tree | c4d0e0cf3ac8a45d1066197f696a26394d476152 /arch/arm/include/asm/tlb.h | |
parent | fad4f3e5e2f5de45bfac686d8c29ec1900fa2824 (diff) |
arm: mm: implement get_user_pages_fastlinux-lng-preempt-rt-v3.10.14-rt9-finallinux-lng-preempt-rt-v3.10.14-rt9linux-lng-preempt-rt-3.10.14-2013.10linux-lng-preempt-rt-v3.10.14-rt9
An implementation of get_user_pages_fast for ARM. It is based loosely
on the PowerPC implementation, but has a few subtle differences.
Under other architectures, the get_user_pages_fast implementations
disable the IRQs in the critical section. This protects against pages
backing page tables from being freed and from THP splits occurring as
these both call TLB invalidations which trigger an IPI which blocks
until the IRQs are re-enabled in get_user_page_fast.
Under ARM, TLB invalidations are usually broadcast in hardware thus
obviating the need for an IPI. After some discussion with Will Deacon:
http://marc.info/?l=linux-mm&m=138089480306901&w=2
It was decided that atomics should be used to protect the critical
section in get_user_pages_fast.
Calls to get_user_pages_fast, cause an atomic, gup_readers, to be
incremented. This guarantees that pages backing page tables won't be
freed from under it as both pte_free and tlb_flush_mmu will block on
non-zero values of gup_readers. Also, this guarantees that THPs will
not split, as an implementation of pmdp_splitting_flush is provided
that also blocks on non-zero values of gup_readers.
Signed-off-by: Steve Capper <steve.capper@linaro.org>
Signed-off-by: Kim Phillips <kim.phillips@linaro.org>
Diffstat (limited to 'arch/arm/include/asm/tlb.h')
-rw-r--r-- | arch/arm/include/asm/tlb.h | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/arch/arm/include/asm/tlb.h b/arch/arm/include/asm/tlb.h index 0baf7f0d9394..470ef9e1e697 100644 --- a/arch/arm/include/asm/tlb.h +++ b/arch/arm/include/asm/tlb.h @@ -100,6 +100,14 @@ static inline void __tlb_alloc_page(struct mmu_gather *tlb) static inline void tlb_flush_mmu(struct mmu_gather *tlb) { + /* + * Before freeing pages, check to see whether or not + * __get_user_pages_fast is still walking pages in the mm. + * If this is the case, wait until gup has finished. + */ + while (atomic_read(&tlb->mm->context.gup_readers) != 0) + cpu_relax(); + tlb_flush(tlb); free_pages_and_swap_cache(tlb->pages, tlb->nr); tlb->nr = 0; |