Skip to main content
Version: 2.34.0

Load Scheduling for Elasticsearch

Introduction

By default, this policy detects when the Elasticsearch service is overloaded using the search thread pool queue size metric. The policy is based on the adaptive load scheduling component.

All the Elasticsearch related metrics are collected by the Elasticsearch OpenTelemetry Collector so if the system under observation requires using different metrics for the overload confirmation, the list of available metrics can be used to configure the policy. The following PromQL query (with appropriate filters) is used as SIGNAL for the load scheduler:

avg(elasticsearch_node_thread_pool_tasks_queued)

Configuration

Blueprint name: load-scheduling/elasticsearch

Parameters

policy

Parameterpolicy.components
DescriptionList of additional circuit components.
TypeArray of Object (aperture.spec.v1.Component)
Default Value
Expand
[]
Parameterpolicy.policy_name
DescriptionName of the policy.
Typestring
Default Value__REQUIRED_FIELD__
Parameterpolicy.resources
DescriptionAdditional resources.
TypeObject (aperture.spec.v1.Resources)
Default Value
Expand
flow_control:
classifiers: []
Parameterpolicy.elasticsearch
DescriptionConfiguration for Elasticsearch OpenTelemetry receiver. Refer https://docs.fluxninja.com/integrations/metrics/elasticsearch for more information.
TypeObject (elasticsearch)
Default Value
Expand
agent_group: default
endpoint: __REQUIRED_FIELD__
password: __REQUIRED_FIELD__
username: __REQUIRED_FIELD__
Parameterpolicy.search_queue_threshold
DescriptionThreshold of search thread pool tasks queued at which point to start throttling traffic.
TypeNumber (double)
Default Value__REQUIRED_FIELD__
policy.load_scheduling_core
Parameterpolicy.load_scheduling_core.dry_run
DescriptionDefault configuration for setting dry run mode on Load Scheduler. In dry run mode, the Load Scheduler acts as a passthrough and does not throttle flows. This config can be updated at runtime without restarting the policy.
TypeBoolean
Default Valuefalse
Parameterpolicy.load_scheduling_core.kubelet_overload_confirmations
DescriptionOverload confirmation signals from kubelet.
TypeObject (kubelet_overload_confirmations)
Default Value
Expand
{}
Parameterpolicy.load_scheduling_core.overload_confirmations
DescriptionList of overload confirmation criteria. Load scheduler can throttle flows when all of the specified overload confirmation criteria are met.
TypeArray of Object (overload_confirmation)
Default Value
Expand
[]
Parameterpolicy.load_scheduling_core.aiad_load_scheduler
DescriptionParameters for AIMD throttling strategy.
TypeObject (aperture.spec.v1.AIADLoadSchedulerParameters)
Default Value
Expand
alerter:
alert_name: AIAD Load Throttling Event
load_multiplier_linear_decrement: 0.05
load_multiplier_linear_increment: 0.025
load_scheduler:
selectors:
- control_point: __REQUIRED_FIELD__
max_load_multiplier: 2
min_load_multiplier: 0

Schemas

driver_criteria

Parameterenabled
DescriptionEnables the driver.
TypeBoolean
Default Value__REQUIRED_FIELD__
Parameterthreshold
DescriptionThreshold for the driver.
TypeNumber (double)
Default Value__REQUIRED_FIELD__

overload_confirmation_driver

Parameterpod_cpu
DescriptionThe driver for using CPU usage as overload confirmation.
TypeObject (driver_criteria)
Default Value
Expand
{}
Parameterpod_memory
DescriptionThe driver for using memory usage as overload confirmation.
TypeObject (driver_criteria)
Default Value
Expand
{}

kubelet_overload_confirmations

Parametercriteria
DescriptionCriteria for overload confirmation.
TypeObject (overload_confirmation_driver)
Default Value__REQUIRED_FIELD__
Parameterinfra_context
DescriptionKubernetes selector for scraping metrics.
TypeObject (aperture.spec.v1.KubernetesObjectSelector)
Default Value__REQUIRED_FIELD__

overload_confirmation

Parameteroperator
DescriptionThe operator for the overload confirmation criteria. oneof: `gt | lt | gte | lte | eq | neq`
Typestring
Default Value
Parameterquery_string
DescriptionThe Prometheus query to be run. Must return a scalar or a vector with a single element.
Typestring
Default Value
Parameterthreshold
DescriptionThe threshold for the overload confirmation criteria.
TypeNumber (double)
Default Value

elasticsearch

Parameteragent_group
DescriptionName of the Aperture Agent group.
Typestring
Default Valuedefault
Parameterendpoint
DescriptionEndpoint of the Elasticsearch.
Typestring
Default Value__REQUIRED_FIELD__
Parameterindices
DescriptionIndex filters that define which indices are scraped for index-level metrics.
TypeArray of string
Default Value
Parameterinitial_delay
DescriptionDefines how long this receiver waits before starting.
Typestring
Default Value
Parameternodes
DescriptionNode filters that define which nodes are scraped for node-level and cluster-level metrics.
TypeArray of string
Default Value
Parameterpassword
DescriptionPassword of the Elasticsearch.
Typestring
Default Value__REQUIRED_FIELD__
Parameterskip_cluster_metrics
DescriptionIf true, cluster-level metrics will not be scraped.
TypeBoolean
Default Value
Parameterusername
DescriptionUsername of the Elasticsearch.
Typestring
Default Value__REQUIRED_FIELD__

Dynamic Configuration

note

The following configuration parameters can be dynamically configured at runtime, without reloading the policy.

Parameters

Parameterdry_run
DescriptionDynamic configuration for setting dry run mode at runtime without restarting this policy. In dry run mode the scheduler acts as pass through to all flow and does not queue flows. It is useful for observing the behavior of load scheduler without disrupting any real traffic.
TypeBoolean
Default Value__REQUIRED_FIELD__