diff options
author | Catalin Marinas <catalin.marinas@arm.com> | 2012-02-23 17:16:41 +0000 |
---|---|---|
committer | Jon Medhurst <tixy@linaro.org> | 2012-07-09 14:26:12 +0100 |
commit | cb2f559a624648da125cb7934d35e16c24532dbd (patch) | |
tree | 00288c1384237dab2ec6663428010a6114d007cb | |
parent | 44d9f6eec5da8f6b9341ebd7cc071c8a514ef656 (diff) |
ARM: LPAE: Invalidate the TLB for module addresses during translation fault
During the free_pgtables() call all user and modules/pkmap entries are
removed. If a translation fault for the modules/pkmap area occurs before
we switched away from the current pgd, do_translation_fault() would copy
the init_mm pud into the user pud.
There is a small window between pud_clear() and pmd_free_tlb() in
free_pmd_range() where the pud entry was cleared but the TLB has not
been invalidated yet and the CPU may have cached the original (valid)
pud entry in the TLB. A scenario like below would get stuck in
continuous prefetch abort:
1. Current process exiting. The modules pmd entries not populated
2. exit_mmap() -> ... -> pmd_free_tlb()
3. pud_clear() for the 1GB pud containing user stack and modules (no TLB
invalidation yet)
4. Interrupt -> module interrupt routine
5. Level 2 (pmd) translation fault occurs when executing the module
interrupt routine. The CPU previously cached (TLB) the old valid pud
value for the modules area, so we don't get a level 1 translation
fault
6. do_translation fault() copies the pud_k into the pud
7. Linux returns to the faulty instruction. Goes back to 5
At point 7, since the CPU still has the old pud value, it goes back to
point 5 and never gets out of this loop. With this patch, the stale pud
TLB entry is invalidated after point 6 and the subsequent prefetch abort
doesn't occur.
Reported-by: Tony Thompson <Anthony.Thompson@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
-rw-r--r-- | arch/arm/mm/fault.c | 13 |
1 files changed, 11 insertions, 2 deletions
diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index c3bd83450227..44df3649dc13 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -445,8 +445,16 @@ do_translation_fault(unsigned long addr, unsigned int fsr, if (pud_none(*pud_k)) goto bad_area; - if (!pud_present(*pud)) + if (!pud_present(*pud)) { set_pud(pud, *pud_k); + /* + * There is a small window during free_pgtables() where the + * user *pud entry is 0 but the TLB has not been invalidated + * and we get a level 2 (pmd) translation fault caused by the + * intermediate TLB caching of the old level 1 (pud) entry. + */ + flush_tlb_kernel_page(addr); + } pmd = pmd_offset(pud, addr); pmd_k = pmd_offset(pud_k, addr); @@ -469,8 +477,9 @@ do_translation_fault(unsigned long addr, unsigned int fsr, #endif if (pmd_none(pmd_k[index])) goto bad_area; + if (!pmd_present(pmd[index])) + copy_pmd(pmd, pmd_k); - copy_pmd(pmd, pmd_k); return 0; bad_area: |