Thomas Gleixner
b1773eac3f
It has become an obsession to mitigate the determinism vs. throughput loss of RT. Looking at the mainline semantics of preemption points gives a hint why RT sucks throughput wise for ordinary SCHED_OTHER tasks. One major issue is the wakeup of tasks which are right away preempting the waking task while the waking task holds a lock on which the woken task will block right after having preempted the wakee. In mainline this is prevented due to the implicit preemption disable of spin/rw_lock held regions. On RT this is not possible due to the fully preemptible nature of sleeping spinlocks. Though for a SCHED_OTHER task preempting another SCHED_OTHER task this is really not a correctness issue. RT folks are concerned about SCHED_FIFO/RR tasks preemption and not about the purely fairness driven SCHED_OTHER preemption latencies. So I introduced a lazy preemption mechanism which only applies to SCHED_OTHER tasks preempting another SCHED_OTHER task. Aside of the existing preempt_count each tasks sports now a preempt_lazy_count which is manipulated on lock acquiry and release. This is slightly incorrect as for lazyness reasons I coupled this on migrate_disable/enable so some other mechanisms get the same treatment (e.g. get_cpu_light). Now on the scheduler side instead of setting NEED_RESCHED this sets NEED_RESCHED_LAZY in case of a SCHED_OTHER/SCHED_OTHER preemption and therefor allows to exit the waking task the lock held region before the woken task preempts. That also works better for cross CPU wakeups as the other side can stay in the adaptive spinning loop. For RT class preemption there is no change. This simply sets NEED_RESCHED and forgoes the lazy preemption counter. Initial test do not expose any observable latency increasement, but history shows that I've been proven wrong before :) The lazy preemption mode is per default on, but with CONFIG_SCHED_DEBUG enabled it can be disabled via: # echo NO_PREEMPT_LAZY >/sys/kernel/debug/sched_features and reenabled via # echo PREEMPT_LAZY >/sys/kernel/debug/sched_features The test results so far are very machine and workload dependent, but there is a clear trend that it enhances the non RT workload performance. Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
143 lines
4.9 KiB
Plaintext
143 lines
4.9 KiB
Plaintext
# SPDX-License-Identifier: GPL-2.0-only
|
|
|
|
config HAVE_PREEMPT_LAZY
|
|
bool
|
|
|
|
config PREEMPT_LAZY
|
|
def_bool y if HAVE_PREEMPT_LAZY && PREEMPT_RT
|
|
|
|
config PREEMPT_NONE_BUILD
|
|
bool
|
|
|
|
config PREEMPT_VOLUNTARY_BUILD
|
|
bool
|
|
|
|
config PREEMPT_BUILD
|
|
bool
|
|
select PREEMPTION
|
|
select UNINLINE_SPIN_UNLOCK if !ARCH_INLINE_SPIN_UNLOCK
|
|
|
|
choice
|
|
prompt "Preemption Model"
|
|
default PREEMPT_NONE
|
|
|
|
config PREEMPT_NONE
|
|
bool "No Forced Preemption (Server)"
|
|
select PREEMPT_NONE_BUILD if !PREEMPT_DYNAMIC
|
|
help
|
|
This is the traditional Linux preemption model, geared towards
|
|
throughput. It will still provide good latencies most of the
|
|
time, but there are no guarantees and occasional longer delays
|
|
are possible.
|
|
|
|
Select this option if you are building a kernel for a server or
|
|
scientific/computation system, or if you want to maximize the
|
|
raw processing power of the kernel, irrespective of scheduling
|
|
latencies.
|
|
|
|
config PREEMPT_VOLUNTARY
|
|
bool "Voluntary Kernel Preemption (Desktop)"
|
|
depends on !ARCH_NO_PREEMPT
|
|
select PREEMPT_VOLUNTARY_BUILD if !PREEMPT_DYNAMIC
|
|
help
|
|
This option reduces the latency of the kernel by adding more
|
|
"explicit preemption points" to the kernel code. These new
|
|
preemption points have been selected to reduce the maximum
|
|
latency of rescheduling, providing faster application reactions,
|
|
at the cost of slightly lower throughput.
|
|
|
|
This allows reaction to interactive events by allowing a
|
|
low priority process to voluntarily preempt itself even if it
|
|
is in kernel mode executing a system call. This allows
|
|
applications to run more 'smoothly' even when the system is
|
|
under load.
|
|
|
|
Select this if you are building a kernel for a desktop system.
|
|
|
|
config PREEMPT
|
|
bool "Preemptible Kernel (Low-Latency Desktop)"
|
|
depends on !ARCH_NO_PREEMPT
|
|
select PREEMPT_BUILD
|
|
help
|
|
This option reduces the latency of the kernel by making
|
|
all kernel code (that is not executing in a critical section)
|
|
preemptible. This allows reaction to interactive events by
|
|
permitting a low priority process to be preempted involuntarily
|
|
even if it is in kernel mode executing a system call and would
|
|
otherwise not be about to reach a natural preemption point.
|
|
This allows applications to run more 'smoothly' even when the
|
|
system is under load, at the cost of slightly lower throughput
|
|
and a slight runtime overhead to kernel code.
|
|
|
|
Select this if you are building a kernel for a desktop or
|
|
embedded system with latency requirements in the milliseconds
|
|
range.
|
|
|
|
config PREEMPT_RT
|
|
bool "Fully Preemptible Kernel (Real-Time)"
|
|
depends on EXPERT && ARCH_SUPPORTS_RT
|
|
select PREEMPTION
|
|
help
|
|
This option turns the kernel into a real-time kernel by replacing
|
|
various locking primitives (spinlocks, rwlocks, etc.) with
|
|
preemptible priority-inheritance aware variants, enforcing
|
|
interrupt threading and introducing mechanisms to break up long
|
|
non-preemptible sections. This makes the kernel, except for very
|
|
low level and critical code paths (entry code, scheduler, low
|
|
level interrupt handling) fully preemptible and brings most
|
|
execution contexts under scheduler control.
|
|
|
|
Select this if you are building a kernel for systems which
|
|
require real-time guarantees.
|
|
|
|
endchoice
|
|
|
|
config PREEMPT_COUNT
|
|
bool
|
|
|
|
config PREEMPTION
|
|
bool
|
|
select PREEMPT_COUNT
|
|
|
|
config PREEMPT_DYNAMIC
|
|
bool "Preemption behaviour defined on boot"
|
|
depends on HAVE_PREEMPT_DYNAMIC && !PREEMPT_RT
|
|
select JUMP_LABEL if HAVE_PREEMPT_DYNAMIC_KEY
|
|
select PREEMPT_BUILD
|
|
default y if HAVE_PREEMPT_DYNAMIC_CALL
|
|
help
|
|
This option allows to define the preemption model on the kernel
|
|
command line parameter and thus override the default preemption
|
|
model defined during compile time.
|
|
|
|
The feature is primarily interesting for Linux distributions which
|
|
provide a pre-built kernel binary to reduce the number of kernel
|
|
flavors they offer while still offering different usecases.
|
|
|
|
The runtime overhead is negligible with HAVE_STATIC_CALL_INLINE enabled
|
|
but if runtime patching is not available for the specific architecture
|
|
then the potential overhead should be considered.
|
|
|
|
Interesting if you want the same pre-built kernel should be used for
|
|
both Server and Desktop workloads.
|
|
|
|
config SCHED_CORE
|
|
bool "Core Scheduling for SMT"
|
|
depends on SCHED_SMT
|
|
help
|
|
This option permits Core Scheduling, a means of coordinated task
|
|
selection across SMT siblings. When enabled -- see
|
|
prctl(PR_SCHED_CORE) -- task selection ensures that all SMT siblings
|
|
will execute a task from the same 'core group', forcing idle when no
|
|
matching task is found.
|
|
|
|
Use of this feature includes:
|
|
- mitigation of some (not all) SMT side channels;
|
|
- limiting SMT interference to improve determinism and/or performance.
|
|
|
|
SCHED_CORE is default disabled. When it is enabled and unused,
|
|
which is the likely usage by Linux distributions, there should
|
|
be no measurable impact on performance.
|
|
|
|
|