summaryrefslogtreecommitdiffstats
path: root/arch/arm64/kernel/irq.c (follow)
Commit message (Collapse)AuthorAgeFilesLines
* arm64: Add vmap_stack header fileJames Morse2018-01-131-11/+2
| | | | | | | | | | | | | | Today the arm64 arch code allocates an extra IRQ stack per-cpu. If we also have SDEI and VMAP stacks we need two extra per-cpu VMAP stacks. Move the VMAP stack allocation out to a helper in a new header file. This avoids missing THREADINFO_GFP, or getting the all-important alignment wrong. Signed-off-by: James Morse <james.morse@arm.com> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
* arm64: add basic VMAP_STACK supportMark Rutland2017-08-151-2/+28
| | | | | | | | | | | | | | | | | | | | This patch enables arm64 to be built with vmap'd task and IRQ stacks. As vmap'd stacks are mapped at page granularity, stacks must be a multiple of PAGE_SIZE. This means that a 64K page kernel must use stacks of at least 64K in size. To minimize the increase in Image size, IRQ stacks are dynamically allocated at boot time, rather than embedding the boot CPU's IRQ stack in the kernel image. This patch was co-authored by Ard Biesheuvel and Mark Rutland. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Will Deacon <will.deacon@arm.com> Tested-by: Laura Abbott <labbott@redhat.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: James Morse <james.morse@arm.com>
* arm64: use an irq stack pointerMark Rutland2017-08-151-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | We allocate our IRQ stacks using a percpu array. This allows us to generate our IRQ stack pointers with adr_this_cpu, but bloats the kernel Image with the boot CPU's IRQ stack. Additionally, these are packed with other percpu variables, and aren't guaranteed to have guard pages. When we enable VMAP_STACK we'll want to vmap our IRQ stacks also, in order to provide guard pages and to permit more stringent alignment requirements. Doing so will require that we use a percpu pointer to each IRQ stack, rather than allocating a percpu IRQ stack in the kernel image. This patch updates our IRQ stack code to use a percpu pointer to the base of each IRQ stack. This will allow us to change the way the stack is allocated with minimal changes elsewhere. In some cases we may try to backtrace before the IRQ stack pointers are initialised, so on_irq_stack() is updated to account for this. In testing with cyclictest, there was no measureable difference between using adr_this_cpu (for irq_stack) and ldr_this_cpu (for irq_stack_ptr) in the IRQ entry path. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Will Deacon <will.deacon@arm.com> Tested-by: Laura Abbott <labbott@redhat.com> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: James Morse <james.morse@arm.com>
* arm64: remove irq_count and do_softirq_own_stack()James Morse2015-12-211-37/+1
| | | | | | | | | | | | | | | | | | | | | | sysrq_handle_reboot() re-enables interrupts while on the irq stack. The irq_stack implementation wrongly assumed this would only ever happen via the softirq path, allowing it to update irq_count late, in do_softirq_own_stack(). This means if an irq occurs in sysrq_handle_reboot(), during emergency_restart() the stack will be corrupted, as irq_count wasn't updated. Lose the optimisation, and instead of moving the adding/subtracting of irq_count into irq_stack_entry/irq_stack_exit, remove it, and compare sp_el0 (struct thread_info) with sp & ~(THREAD_SIZE - 1). This tells us if we are on a task stack, if so, we can safely switch to the irq stack. Finally, remove do_softirq_own_stack(), we don't need it anymore. Reported-by: Will Deacon <will.deacon@arm.com> Signed-off-by: James Morse <james.morse@arm.com> [will: use get_thread_info macro] Signed-off-by: Will Deacon <will.deacon@arm.com>
* arm64: Add do_softirq_own_stack() and enable irq_stacksJames Morse2015-12-081-1/+37
| | | | | | | | | | | | | | | | | entry.S is modified to switch to the per_cpu irq_stack during el{0,1}_irq. irq_count is used to detect recursive interrupts on the irq_stack, it is updated late by do_softirq_own_stack(), when called on the irq_stack, before __do_softirq() re-enables interrupts to process softirqs. do_softirq_own_stack() is added by this patch, but does not yet switch stack. This patch adds the dummy stack frame and data needed by the previous stack tracing patches. Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: James Morse <james.morse@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
* arm64: Modify stack trace and dump for use with irq_stackAKASHI Takahiro2015-12-081-0/+3
| | | | | | | | | | | | | | | | | | This patch allows unwind_frame() to traverse from interrupt stack to task stack correctly. It requires data from a dummy stack frame, created during irq_stack_entry(), added by a later patch. A similar approach is taken to modify dump_backtrace(), which expects to find struct pt_regs underneath any call to functions marked __exception. When on an irq_stack, the struct pt_regs is stored on the old task stack, the location of which is stored in the dummy stack frame. Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org> [james.morse: merged two patches, reworked for per_cpu irq_stacks, and no alignment guarantees, added irq_stack definitions] Signed-off-by: James Morse <james.morse@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
* arm64: fix a migrating irq bug when hotplug cpuYang Yingliang2015-10-091-62/+0
| | | | | | | | | | | | | | | | | | | | | When cpu is disabled, all irqs will be migratged to another cpu. In some cases, a new affinity is different, the old affinity need to be updated and if irq_set_affinity's return value is IRQ_SET_MASK_OK_DONE, the old affinity can not be updated. Fix it by using irq_do_set_affinity. And migrating interrupts is a core code matter, so use the generic function irq_migrate_all_off_this_cpu() to migrate interrupts in kernel/irq/migration.c. Cc: Jiang Liu <jiang.liu@linux.intel.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Russell King - ARM Linux <linux@arm.linux.org.uk> Cc: Hanjun Guo <hanjun.guo@linaro.org> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Yang Yingliang <yangyingliang@huawei.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
* arm64: force CONFIG_SMP=y and remove redundant #ifdefsWill Deacon2015-07-271-2/+0
| | | | | | | | | | | Nobody seems to be producing !SMP systems anymore, so this is just becoming a source of kernel bugs, particularly if people want to use coherent DMA with non-shared pages. This patch forces CONFIG_SMP=y for arm64, removing a modest amount of code in the process. Signed-off-by: Will Deacon <will.deacon@arm.com>
* ARM64/irq: Use access helper irq_data_get_affinity_mask()Jiang Liu2015-07-221-2/+2
| | | | | | | | | | This is a preparatory patch for moving irq_data struct members. Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com> Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org> Cc: linux-arm-kernel@lists.infradead.org Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
* arm64: Treat handle_arch_irq as a function pointerLaura Abbott2014-11-251-0/+2
| | | | | | | | | | | | | | | | handle_arch_irq isn't actually text, it's just a function pointer. It doesn't need to be stored in the text section and doing so causes problesm if we ever want to make the kernel text read only. Declare handle_arch_irq as a proper function pointer stored in the data section. Reviewed-by: Kees Cook <keescook@chromium.org> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Tested-by: Mark Rutland <mark.rutland@arm.com> Tested-by: Kees Cook <keescook@chromium.org> Signed-off-by: Laura Abbott <lauraa@codeaurora.org> Signed-off-by: Will Deacon <will.deacon@arm.com>
* Merge branch 'irq-core-for-linus' of ↵Linus Torvalds2014-10-091-27/+0
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip Pull irq updates from Thomas Gleixner: "The irq departement delivers: - a cleanup series to get rid of mindlessly copied code. - another bunch of new pointlessly different interrupt chip drivers. Adding homebrewn irq chips (and timers) to SoCs must provide a value add which is beyond the imagination of mere mortals. - the usual SoC irq controller updates, IOW my second cat herding project" * 'irq-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (44 commits) irqchip: gic-v3: Implement CPU PM notifier irqchip: gic-v3: Refactor gic_enable_redist to support both enabling and disabling irqchip: renesas-intc-irqpin: Add minimal runtime PM support irqchip: renesas-intc-irqpin: Add helper variable dev = &pdev->dev irqchip: atmel-aic5: Add sama5d4 support irqchip: atmel-aic5: The sama5d3 has 48 IRQs Documentation: bcm7120-l2: Add Broadcom BCM7120-style L2 binding irqchip: bcm7120-l2: Add Broadcom BCM7120-style Level 2 interrupt controller irqchip: renesas-irqc: Add binding docs for new R-Car Gen2 SoCs irqchip: renesas-irqc: Add DT binding documentation irqchip: renesas-intc-irqpin: Document SoC-specific bindings openrisc: Get rid of handle_IRQ arm64: Get rid of handle_IRQ ARM: omap2: irq: Convert to handle_domain_irq ARM: imx: tzic: Convert to handle_domain_irq ARM: imx: avic: Convert to handle_domain_irq irqchip: or1k-pic: Convert to handle_domain_irq irqchip: atmel-aic5: Convert to handle_domain_irq irqchip: atmel-aic: Convert to handle_domain_irq irqchip: gic-v3: Convert to handle_domain_irq ...
| * arm64: Get rid of handle_IRQMarc Zyngier2014-09-031-11/+0
| | | | | | | | | | | | | | | | | | | | All the arm64 irqchip drivers have been converted to handle_domain_irq, making it possible to remove the handle_IRQ stub entierely. Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Link: https://lkml.kernel.org/r/1409047421-27649-26-git-send-email-marc.zyngier@arm.com Signed-off-by: Jason Cooper <jason@lakedaemon.net>
| * arm64: Convert handle_IRQ to use __handle_domain_irqMarc Zyngier2014-09-031-17/+1
| | | | | | | | | | | | | | | | | | | | In order to limit code duplication, convert the architecture specific handle_IRQ to use the generic __handle_domain_irq function. Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Link: https://lkml.kernel.org/r/1409047421-27649-3-git-send-email-marc.zyngier@arm.com Signed-off-by: Jason Cooper <jason@lakedaemon.net>
* | arm64: use irq_set_affinity with force=false when migrating irqsSudeep Holla2014-09-031-8/+4
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The arm64 interrupt migration code on cpu offline calls irqchip.irq_set_affinity() with the argument force=true. Originally this argument had no effect because it was not used by any interrupt chip driver and there was no semantics defined. This changed with commit 01f8fa4f01d8 ("genirq: Allow forcing cpu affinity of interrupts") which made the force argument useful to route interrupts to not yet online cpus without checking the target cpu against the cpu online mask. The following commit ffde1de64012 ("irqchip: gic: Support forced affinity setting") implemented this for the GIC interrupt controller. As a consequence the cpu offline irq migration fails if CPU0 is offlined, because CPU0 is still set in the affinity mask and the validation against cpu online mask is skipped to the force argument being true. The following first_cpu(mask) selection always selects CPU0 as the target. Commit 601c942176d8("arm64: use cpu_online_mask when using forced irq_set_affinity") intended to fix the above mentioned issue but introduced another issue where affinity can be migrated to a wrong CPU due to unconditional copy of cpu_online_mask. As with for arm, solve the issue by calling irq_set_affinity() with force=false from the CPU offline irq migration code so the GIC driver validates the affinity mask against CPU online mask and therefore removes CPU0 from the possible target candidates. Also revert the changes done in the commit 601c942176d8 as it's no longer needed. Tested on Juno platform. Fixes: 601c942176d8("arm64: use cpu_online_mask when using forced irq_set_affinity") Signed-off-by: Sudeep Holla <sudeep.holla@arm.com> Acked-by: Mark Rutland <mark.rutland@arm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will.deacon@arm.com> Cc: <stable@vger.kernel.org> # 3.10.x Signed-off-by: Will Deacon <will.deacon@arm.com>
* arm64: use cpu_online_mask when using forced irq_set_affinitySudeep Holla2014-05-121-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 01f8fa4f01d8("genirq: Allow forcing cpu affinity of interrupts") enabled the forced irq_set_affinity which previously refused to route an interrupt to an offline cpu. Commit ffde1de64012("irqchip: Gic: Support forced affinity setting") implements this force logic and disables the cpu online check for GIC interrupt controller. When __cpu_disable calls migrate_irqs, it disables the current cpu in cpu_online_mask and uses forced irq_set_affinity to migrate the IRQs away from the cpu but passes affinity mask with the cpu being offlined also included in it. When calling irq_set_affinity with force == true in a cpu hotplug path, the caller must ensure that the cpu being offlined is not present in the affinity mask or it may be selected as the target CPU, leading to the interrupt not being migrated. This patch uses cpu_online_mask when using forced irq_set_affinity so that the IRQs are properly migrated away. Signed-off-by: Sudeep Holla <sudeep.holla@arm.com> Acked-by: Mark Rutland <mark.rutland@arm.com> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
* arm64: add CPU_HOTPLUG infrastructureMark Rutland2013-10-251-0/+61
| | | | | | | | | This patch adds the basic infrastructure necessary to support CPU_HOTPLUG on arm64, based on the arm implementation. Actual hotplug support will depend on an implementation's cpu_operations (e.g. PSCI). Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
* arm64: Use irqchip_init() for interrupt controller initialisationCatalin Marinas2013-03-261-10/+9
| | | | | | | | This patch uses the generic irqchip_init() function for initialising the interrupt controller on arm64. It also adds several definitions required by the ARM GIC irqchip driver but does not enable ARM_GIC yet. Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
* arm64: IRQ handlingMarc Zyngier2012-09-171-0/+84
This patch adds the support for IRQ handling. The actual interrupt controller will be part of a separate patch (going into drivers/irqchip/). Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Tony Lindgren <tony@atomide.com> Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Olof Johansson <olof@lixom.net> Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>