summaryrefslogtreecommitdiffstats
path: root/kernel/sched/smp.h
diff options
context:
space:
mode:
authorValentin Schneider <vschneid@redhat.com>2023-03-07 15:35:58 +0100
committerPeter Zijlstra <peterz@infradead.org>2023-03-24 11:01:29 +0100
commit68f4ff04dbada18dad79659c266a8e5e29e458cd (patch)
tree38355c3d6f8f5aed482a7e7ba1a3a54117745f39 /kernel/sched/smp.h
parentsmp: reword smp call IPI comment (diff)
downloadlinux-68f4ff04dbada18dad79659c266a8e5e29e458cd.tar.xz
linux-68f4ff04dbada18dad79659c266a8e5e29e458cd.zip
sched, smp: Trace smp callback causing an IPI
Context ======= The newly-introduced ipi_send_cpumask tracepoint has a "callback" parameter which so far has only been fed with NULL. While CSD_TYPE_SYNC/ASYNC and CSD_TYPE_IRQ_WORK share a similar backing struct layout (meaning their callback func can be accessed without caring about the actual CSD type), CSD_TYPE_TTWU doesn't even have a function attached to its struct. This means we need to check the type of a CSD before eventually dereferencing its associated callback. This isn't as trivial as it sounds: the CSD type is stored in __call_single_node.u_flags, which get cleared right before the callback is executed via csd_unlock(). This implies checking the CSD type before it is enqueued on the call_single_queue, as the target CPU's queue can be flushed before we get to sending an IPI. Furthermore, send_call_function_single_ipi() only has a CPU parameter, and would need to have an additional argument to trickle down the invoked function. This is somewhat silly, as the extra argument will always be pushed down to the function even when nothing is being traced, which is unnecessary overhead. Changes ======= send_call_function_single_ipi() is only used by smp.c, and is defined in sched/core.c as it contains scheduler-specific ops (set_nr_if_polling() of a CPU's idle task). Split it into two parts: the scheduler bits remain in sched/core.c, and the actual IPI emission is moved into smp.c. This lets us define an __always_inline helper function that can take the related callback as parameter without creating useless register pressure in the non-traced path which only gains a (disabled) static branch. Do the same thing for the multi IPI case. Signed-off-by: Valentin Schneider <vschneid@redhat.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/20230307143558.294354-8-vschneid@redhat.com
Diffstat (limited to 'kernel/sched/smp.h')
-rw-r--r--kernel/sched/smp.h2
1 files changed, 1 insertions, 1 deletions
diff --git a/kernel/sched/smp.h b/kernel/sched/smp.h
index 2eb23dd0f285..21ac44428bb0 100644
--- a/kernel/sched/smp.h
+++ b/kernel/sched/smp.h
@@ -6,7 +6,7 @@
extern void sched_ttwu_pending(void *arg);
-extern void send_call_function_single_ipi(int cpu);
+extern bool call_function_single_prep_ipi(int cpu);
#ifdef CONFIG_SMP
extern void flush_smp_call_function_queue(void);