Skip to main content
Version: development

CPU Overload Protection (Kubernetes)

Introduction

By default, this policy detects when the Kubernetes Pod is overloaded using the pod CPU utilization metric. The policy is based on the adaptive load scheduling component.

All the Kubernetes related metrics are collected by the Kubeletstats OpenTelemetry Collector so if the system under observation requires using different metrics for the overload confirmation, the list of available metrics can be used to configure the policy. The following PromQL query (with appropriate filters) is used as SIGNAL for the load scheduler:

avg(k8s_pod_cpu_utilization_ratio)

Configuration

Blueprint name: load-scheduling/cpu-overload-protection-k8s

Parameters

policy

Parameterpolicy.components
DescriptionList of additional circuit components.
TypeArray of Object (aperture.spec.v1.Component)
Default Value
Expand
[]
Parameterpolicy.policy_name
DescriptionName of the policy.
Typestring
Default Value__REQUIRED_FIELD__
Parameterpolicy.resources
DescriptionAdditional resources.
TypeObject (aperture.spec.v1.Resources)
Default Value
Expand
flow_control:
classifiers: []
policy.load_scheduling_core
Parameterpolicy.load_scheduling_core.dry_run
DescriptionDefault configuration for setting dry run mode on Load Scheduler. In dry run mode, the Load Scheduler acts as a passthrough and does not throttle flows. This config can be updated at runtime without restarting the policy.
TypeBoolean
Default Valuefalse
Parameterpolicy.load_scheduling_core.kubelet_overload_confirmations
DescriptionOverload confirmation signals from kubelet.
TypeObject (kubelet_overload_confirmations)
Default Value
Expand
{}
Parameterpolicy.load_scheduling_core.overload_confirmations
DescriptionList of overload confirmation criteria. Load scheduler can throttle flows when all of the specified overload confirmation criteria are met.
TypeArray of Object (overload_confirmation)
Default Value
Expand
[]
Parameterpolicy.load_scheduling_core.aiad_load_scheduler
DescriptionParameters for AIMD throttling strategy.
TypeObject (aperture.spec.v1.AIADLoadSchedulerParameters)
Default Value
Expand
alerter:
alert_name: AIAD Load Throttling Event
load_multiplier_linear_decrement: 0.05
load_multiplier_linear_increment: 0.025
load_scheduler:
selectors:
- control_point: __REQUIRED_FIELD__
max_load_multiplier: 2
min_load_multiplier: 0
Parameterpolicy.load_scheduling_core.setpoint
DescriptionSetpoint.
TypeNumber (double)
Default Value__REQUIRED_FIELD__
policy.kubernetes_object_selector
Parameterpolicy.kubernetes_object_selector.api_version
DescriptionAPI version of the object to protect.
Typestring
Default Valueapps/v1
Parameterpolicy.kubernetes_object_selector.kind
DescriptionKind of the object to protect.
Typestring
Default ValueDeployment
Parameterpolicy.kubernetes_object_selector.name
DescriptionName of the object to protect.
Typestring
Default Value__REQUIRED_FIELD__
Parameterpolicy.kubernetes_object_selector.namespace
DescriptionNamespace of the object to protect.
Typestring
Default Value__REQUIRED_FIELD__

Schemas

driver_criteria

Parameterenabled
DescriptionEnables the driver.
TypeBoolean
Default Value__REQUIRED_FIELD__
Parameterthreshold
DescriptionThreshold for the driver.
TypeNumber (double)
Default Value__REQUIRED_FIELD__

overload_confirmation_driver

Parameterpod_cpu
DescriptionThe driver for using CPU usage as overload confirmation.
TypeObject (driver_criteria)
Default Value
Expand
{}
Parameterpod_memory
DescriptionThe driver for using memory usage as overload confirmation.
TypeObject (driver_criteria)
Default Value
Expand
{}

kubelet_overload_confirmations

Parametercriteria
DescriptionCriteria for overload confirmation.
TypeObject (overload_confirmation_driver)
Default Value__REQUIRED_FIELD__
Parameterinfra_context
DescriptionKubernetes selector for scraping metrics.
TypeObject (aperture.spec.v1.KubernetesObjectSelector)
Default Value__REQUIRED_FIELD__

overload_confirmation

Parameteroperator
DescriptionThe operator for the overload confirmation criteria. oneof: `gt | lt | gte | lte | eq | neq`
Typestring
Default Value
Parameterquery_string
DescriptionThe Prometheus query to be run. Must return a scalar or a vector with a single element.
Typestring
Default Value
Parameterthreshold
DescriptionThe threshold for the overload confirmation criteria.
TypeNumber (double)
Default Value

Dynamic Configuration

note

The following configuration parameters can be dynamically configured at runtime, without reloading the policy.

Parameters

Parameterdry_run
DescriptionDynamic configuration for setting dry run mode at runtime without restarting this policy. In dry run mode the scheduler acts as pass through to all flow and does not queue flows. It is useful for observing the behavior of load scheduler without disrupting any real traffic.
TypeBoolean
Default Value__REQUIRED_FIELD__