diff options
| author | Alexandre Ghiti <alexghiti@rivosinc.com> | 2024-02-29 13:10:56 +0100 |
|---|---|---|
| committer | Palmer Dabbelt <palmer@rivosinc.com> | 2024-04-17 03:27:47 +0200 |
| commit | c97bf629963e52b205ed5fbaf151e5bd342f9c63 (patch) | |
| tree | 87bc3f31fc0badf08ea02ae6516ad49055cdcf6c /arch/riscv/kernel/patch.c | |
| parent | riscv: Remove superfluous smp_mb() (diff) | |
| download | linux-c97bf629963e52b205ed5fbaf151e5bd342f9c63.tar.xz linux-c97bf629963e52b205ed5fbaf151e5bd342f9c63.zip | |
riscv: Fix text patching when IPI are used
For now, we use stop_machine() to patch the text and when we use IPIs for
remote icache flushes (which is emitted in patch_text_nosync()), the system
hangs.
So instead, make sure every CPU executes the stop_machine() patching
function and emit a local icache flush there.
Co-developed-by: Björn Töpel <bjorn@rivosinc.com>
Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Reviewed-by: Andrea Parri <parri.andrea@gmail.com>
Link: https://lore.kernel.org/r/20240229121056.203419-3-alexghiti@rivosinc.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Diffstat (limited to 'arch/riscv/kernel/patch.c')
| -rw-r--r-- | arch/riscv/kernel/patch.c | 16 |
1 files changed, 12 insertions, 4 deletions
diff --git a/arch/riscv/kernel/patch.c b/arch/riscv/kernel/patch.c index 0b5c16dfe3f4..9a1bce1adf5a 100644 --- a/arch/riscv/kernel/patch.c +++ b/arch/riscv/kernel/patch.c @@ -188,7 +188,7 @@ int patch_text_set_nosync(void *addr, u8 c, size_t len) } NOKPROBE_SYMBOL(patch_text_set_nosync); -static int patch_insn_write(void *addr, const void *insn, size_t len) +int patch_insn_write(void *addr, const void *insn, size_t len) { size_t patched = 0; size_t size; @@ -232,15 +232,23 @@ static int patch_text_cb(void *data) if (atomic_inc_return(&patch->cpu_count) == num_online_cpus()) { for (i = 0; ret == 0 && i < patch->ninsns; i++) { len = GET_INSN_LENGTH(patch->insns[i]); - ret = patch_text_nosync(patch->addr + i * len, - &patch->insns[i], len); + ret = patch_insn_write(patch->addr + i * len, &patch->insns[i], len); } - atomic_inc(&patch->cpu_count); + /* + * Make sure the patching store is effective *before* we + * increment the counter which releases all waiting CPUs + * by using the release variant of atomic increment. The + * release pairs with the call to local_flush_icache_all() + * on the waiting CPU. + */ + atomic_inc_return_release(&patch->cpu_count); } else { while (atomic_read(&patch->cpu_count) <= num_online_cpus()) cpu_relax(); } + local_flush_icache_all(); + return ret; } NOKPROBE_SYMBOL(patch_text_cb); |