Load Scheduler Reference
The Load Scheduler is used to throttle request rates dynamically during high load, therefore protecting services from overloads and cascading failures. It uses a local token bucket for estimating the allowed token rate. The fill rate of the token bucket gets adjusted by the controller based on the specified policy. Since this component builds upon the Scheduler, it allows defining workloads along with their priority and tokens. The scheduler employs weighted fair queuing of requests to achieve graceful degradation of applications.
This diagram illustrates the working of a load scheduler.
The Load Scheduler's throttling behavior is controlled by the signal at its
load_multiplier input port. As the policy circuit adjusts the signal at the
load multiplier port, it gets translated to the token refill rate at the Agents.
At each Agent, the adjusted token rate is determined by multiplying the past
token rate with the load multiplier. The past 30 seconds of data is used for
finding the past token rate.
If the incoming request rate surpasses the adjusted rate, the scheduler starts queuing requests. The queued requests get admitted as tokens become available in an order determined by the scheduler based on the weighted fair queuing algorithm. Any request that fails to be scheduled within its designated timeout is rejected.
Additionally, by defining workloads with varying priorities and weights (tokens), the load scheduler (and others mentioned below) can prioritize certain requests over others, facilitating graceful service degradation during high-traffic periods.
Additive Increase Multiplicative Decrease Load Scheduler
AIMD Load Scheduler Reference
Offering a more high-level interface, this component has
overload_confirmation ports. The core function of the AIMD Load
Scheduler is its ability to modify the accepted token rate based on the
deviation of the input
signal from the
setpoint. This scheduler reduces
token rate proportionally (or any arbitrary power) based on deviation of the
setpoint. During recovery, it increases the token rate linearly
until the system is not overloaded. It allows for the translation of health
signals into adjustments in token rate, thereby providing an active defense
mechanism for the service.
Additive Increase Additive Decrease Load Scheduler
AIAD Load Scheduler Reference
AIAD Load Scheduler is a high-level circuit component that uses the Load Scheduler internally.
This component has
overload_confirmation ports. It
overload_condition to compare
setpoint to determine if the
service is overloaded. AIAD Load Scheduler reduces the token rate linearly
over time while in overload state. During recovery, it increases the token rate
linearly until the system is not overloaded.
Range-Driven Load Scheduler
Range-Driven Load Scheduler Reference
Range-Driven Load Scheduler is a high-level circuit component that uses the Load Scheduler internally.
This component has
overload_confirmation ports. It uses the
polynomial range function
to throttle the token rate based on the range of the
signal, attempting to
keep it between